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Working Group Summary and Conclusions 

The material in this section summarizes the Working Groups discussions 
held at the workshops detailed in Appendix C of this report. It is impor- 
tant to note that the discussion held by the working group surrounding these 
presentations v/ere far reaching and included some topics not specifically 
presented by individual speakers. As such, v/hile the summary and conclusions 
here are presented with reference to specific workshops some conclusions 
presented summarize essential end-point thinking of working group members 
(i.e., these are conclusions on research issues as of the writing of this 
report) . 

Asilomar Workshop cn Inform.aticn and Decision Processes 
The fact that today's government policy.making process places' increased em- 
phasis on rigorous program evaluation. For example, Presidential Decision 54 
(November 1973) requires that private users of future systems pay product 
prices sufficient to ensure maxim.um recovery of system costs consistent, 
with the public good. Benefits to users must be evaluated against system 
costs. As such future satellite remote sensor systems will develoo only 
if they can be economically justified (i.e., justified in terms of their 
ability to provide data which has potential value as information to users 
which can be effective input into the user decision system (information 
system)). 

Asilomar Workshop participants agreed that no general theory or meth- 
odological framework exists for estimating the value of user decision- 
oriented information systems. A number of factors contribute to this 
methodological problem. First, there is no market price for the output of 


most public information systems, and therefore value cannot be estimated in 
conventional ways. Second, the quality of an information system is based 
on its multiple characteristics, including accuracy, timeliness, reliability, 
continuity, and so on. Moreover, some types of information possess the 
characteristics of a public good (the use of the good by one individual 
does not reduce the amount available to others), and its private value may 
differ substantially from its social value. Third, many sources of infor- 
mation are available to decision-makers, few are regarded as certain, and 
choices among types of information to obtain and use are made within the 
uncertain environments that confront decision-makers. 

Thus, the U'orking Group on Information Utilization and Evaluation has 
concluded that current knowledge and research provide little guidance for 
the evaluation of modifications in the public sector compcner.ts of complex 
user application decision-oriented renewable resource infor-ation systen'.s. 

As previously stated, the future capabilities of alcernacive remote sensing 
systems are all limitless; but the design of future satellite sensor 
systems in a meaningful fashion; however, is quite imperfect. Additionally, 
no generally accepted methodology exists for evaluating changes in infor- 
mation systems, structures, sources, and r’^nducts. Thus, the Working 
Group recommends that an integrated research program be undertaken to: 

(1) Review and document existing renewable resource 


information systems and examine the potential of 
implementing advanced expert system -jpproaches ; 
and. 


(2) Develop the underlying theoretical and methodological 
basis for estimating the value of renewable resource 
data and information systems and their modification. 
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The review and documentation of theoretical and methodological research 
should proceed together and involve some of the same scientists; they are 
complementary endeavors. Thus, research should focus on systems to monitor 
cereal grain production, land cover/use, and soil moisture where remotely 
sensed data are or may be combined with data collected by other means. 

The review and documentation process should not just include systems 

I 

which obviously lend themselves to remote sensing applications. The option 
of combining remotely sensed data with data collected by other means should 
be studied. In fact, this latter possibility indicates v;hy the review 
process should place special emphasis on why, how, and with what effects 
are multiple-sources of data used within existing information systems. 

Uses of remotely sensed data typically involve date end information 
with a wide variety of attributes such as function, scale, timeliness, 
precision, and others. These attributes should be examined in the context 
of systems v/here tradeoffs among specific attributes and institutional 
constraints can be directly addressed. Systematic reviews and documen- 
tation are needed to understand how the attributes of data and information 


derived from it interact and affect analytical and decision processes. 

Most studies on the value of infonr.ation assume, for example, that 
a farmer, an elevator operator, and a buyer all use the same information 
in the same way, but evidence indicates that this is unrealistic. Some 
information is available as a public good, but it is not equally used 
by all who receive it. For these reasons, the identification of major 


market and non-market users of information, their *;ources of information, 
and the adequacy of these sources become important if new systems are to be 
satisfactorily evaluated. Particular attention should be given to the docu- 
mentation of how the behavior of public and private users and producers of 
information are changed when information is viewed as a public rather than 
a private good. 

As future information systems are developed careful attention should 
be given to techniques emerging from the rapidly developing field termed 
"artificial intelligence." Research in this field is addressing new approaches 
to knowledge representation , language understanding, heuristic search, and 
other symbolic reasoning problems directly oertinent to many of the problems 
encountered by paradigms employed in existing information systems (Shortliffe, 
et al ., 1979). 

Analyses of many human decision-making processes suggest that as 
decisions move from simple to complex, the reasoning style becomes less 
algorithmic and more heuristic, while qualitative judgmental knowledge and 
the conditions for invoking it appear to increase. The artificial intel- 
ligence techniques used in "expert systems" more closely mimic this approach 
than previously used paradigms. In proposing research areas for information 
and decision processes. Bacon in a letter to Estes dated 2 June 1981, noted:* 

"1 believe the most likely approach and therefore the 
first priority is that associated with expert systems. 

In order for the general public or laymen "government 
agencies" to obtain value from satellite data, a great 
deal of highly technical assistance must be given. I 
believe the expert system approach would be ideal and 
would make a very useful demonstration project. At the 
same time, this area is considered to be one of the more 
advanced areas of computer science discipline." 


*The entire letter from Bacon to Estes is included in Appendix D. 


Research exploring the linkages and possible use of expert systems 
techniques are needed in at least three •'npertant areas: 

(1 ) Discipline Concepts - 

Traditional paradigms have no true "understanding" of the 
discipline involved. Although explicit decision trees can give 
decision theory programs a greater sense of pertinent associa- 
tions, true discipline knowledge and the heuristics for problem 
solving are not explicitly represented nor used. So-called 
"commonsense" is often clearly lacking when existing systems 
fail, and this is often what most alienates potential users. 

In contrast, expert systems make explicit use of production 
rules that relate observations to associated inferences that may 
be drawn, 

(2) Conversational Cababilities - 

Both for capturing knowledge from collaborating experts 
and for communicating with users. The need is strong for the 
development of computer-based linguistic capabilities. 

(3) Explanation - 

Alternative systems seldom emphasize an ability to explain 
the basis for their decisions in terms understandable to the 
user. This can leave the user with no basis for deciding whether 
or not to accept the system's results and can lead to resentment 
at what could be perceived as an attempt to dictate the decision- 
making process. Once again in contrast, the heuristics of an 
expert system can often form a coherent explanation of system 
reasoning. 
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Working Group merr.bers feel strongly that work is required in this new 
and important approach and that there are potentially great benefits to be 
gained far beyond their application in the information and decision-making 
process area. 

Tevard c-i /eloping the underlying theoi'etical and mefhociological basis 
for estimating the value of renewable resource data and information systems 
top priority is necessary. The long-run goal would be the Formulation of 
quantitative models capable of contributing to the evaluation of trade-offs 
between existing and potential new inforration systems. Such models should 
be able to describe and explain existing systems and to prcvice tne mean.s to 
identify and estimate the magnitude of future consequences of new systems 
with alternative features. Specific research tasks include; 

■* Development of a general theory c-f ere economics o* in-orma- 
tion systems and their modi f i cation , 

* Development of relevant measures for atari outes of caca and 
information (e.g., minimally sufficient statistics) and 
empirical methods for investigating information system per- 
formance. 

* Development of models, at various levels of aggregation, of 
information systems with public, orivate, and international 
components. 

* investigation of analytical systems and processing strategies 
that pertain to the extraction of information from data based 
on disparate concepts. 

* Analyses of data and information dissemination system.s, includ- 
ing studies of accessibility and decentralization, economies 

of scale, confidentiality and property rights, public good 
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fcels that this type of analysis could best be accomplished from a funda- 
mental perspective if examined v/ithin the context of potential implementa- 
tion of expert system approaches to the generation of information from re- 
motely sensed data. 

San Jose Workshop on Data Base Management and Use 
During the San Jose Workshop on Data Base Management and Use it was 
discovered that the compute? science community beyond NASA's direct concern 
are working on improving our ability to handle larger files and perform 
special purpose (e.g., array) processing to increase the speed of image. 

There is currently a major commercial motivation for highly integrated 
computer assisted design and manufacturing systems that require interaction 
with graphical information in a manner similar to the useage of remotely 
sensed image information. General purpose data manipulation and accessing 
facilities for these systems should provide the base for NASA facilities in 
the future. 

The potential exists for major breakthroughs changing the basic 
structure of data base management systems as they exist today. A number of 

Working Group and v/orkshop participants felt thac it is the responsibili ty 

of NASA and the funders of the Workshop (with respect to the Renewable 

Resources Program in particular) to make their .needs under's toed so that the 

computer science cor.munity can focus on imp-'cvement. Today, data base 

management systems are primarily designed to serve the banking/accounting 

community. Special purpose hardware systems are also being developed for 

military application which are near- and real-time oriented. In addition, 

federal agencies are cooperating with the commercial community and the 

American National Standards Institute (ANSI) on developing data handling 

standards and interchange formats. By focusing research on data base 
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management through systems level demonstration projects NASA fundamental 
research program participants should provide a solid framework for the 
Renewable Resources Program to articulate both its immediate and long range 
needs to the research community and the commercial sector. 

As discussed here system level demonstration projects involve the valid- 
ation and testing of data base generation and use scenarios, and can there- 
fore be considered orthogonal to technology and technique development programs 
being pursued by NASA/OAST, the Data Systems Branch within NASA/OSSA, and 
the general communications field. Systems level demonstration projects 
sponsored by the Fundamental Research Program would reveal current and 
potential problems associated with data handling gaps and bottlenecks for 
applications and system tests applied to resource inventory and modeling 
and serve to illuminate the potential for the integration and implementation 
of artificial intelligence/expert system approaches to resource management 
problems. In addition, renewable resources should keep abrest of work in 
the Data Systems Branch on a Transportable Applications Executive (TAE) as 
this concept has important implications for furthering the combined research 
potential of the NASA center. Such fundamental work as TAE can improve NASA's 
overall potential for advancing the application of advanced image processing 
to a wide variety of studies. 

Working Group members feel that in the area of Data Base Use anv. Manage- 
ment that the case study (and scenario) approach is a practical means for 
determining <»ssential components for data base management and use as they 
are applied to earth observations apol ications . We feel three general types 
of case studios, emphasizing present and future sensor platforms, could be 
employed to analyze a range of potential applications. Suggested topics are: 
a) crop yield forecast in a foreign country; b) watershed runoff forecasting; 
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and c) prediction of changes in land use within a region. Each of these 
studies can be thought of as exercising different aspects of data base manage- 
ment technology being developed by the communi coitions and computer indus- 
tries. They will also test the basic assumption that conceptual models 
designed to analyze a process and derive a prediction can in fact incorporate 
satellite sensor data efficiently and effectively to accurately address a problem 
of general concern to users. Each of tlie three suggested case itudy areas are 
being actively researched by government agencies and the private sector today. 

Thus, good bench n-.arks for timelines, accuracy, and level of detail required 
for the end-product exists. The massive amounts, and the specialized require- 
ments for calibration, interpretation, and formatting of satellite sensor 
data have in past NASA application demonstrations become the overriding 
technical concern. As a result, for each case study proposed the following 
elements need to be invest! gatea: 

1) Impacts of data base structure on sensor and ancillary data 
input and archiving formats; 

2) Impacts of and architectures fo»- query capabilities and pro- 
cessing rates on system development and use; 

3) Impact of data availability, arcnivirg, and opportunity 
costs of data storage; and, 

4) Impact of decentralized data bases/informaticn systems on 
cost/benefit ratios for systems use. 

In addition, technology research parallel to that being undertaken in 
other branches of OSSA and OSTA needs to be undertaken to assume the avail- 
ability of viable processing strategies in future systems capabilities to 
integrate disparate data types. Key issues include: 


Understanding; qftneric f unctions in spatially oriented data manipulatiop . 
These include data capture, data registration, and data analysis. Data 
capture involves the encoding of image, vector, and tabular data types 
which represent or are keyed to areas on the ground. Data registration 

involves both the conversion of satellite and ancillary data products 
to a given map projection, and assurance that the information content 

of a data set is properly represented for the map scale used in the 

analysis. Data analysis involves the combination of one or more of the 

following primitive functions: a) Given a point and a district, does 

the point lie within the district, b) Given a point and a district 

file, which district contains the point, c) Given a particular dis- 
trict in a district file, what are its neighbors, d) Given a district 
file and an area cla ssi f’cation file, what are the acreages of each 
area classification ir» each district, e) Given two district files, 
one major and one minor, what are the proportions of each minor dis- 
trict in each major district, f) Given a district file and a line 
segment, what are the mileages of the line segment in each district, 
g) Given a point p and a point file, what is the point in the point 

file nearest to p. h) Given a point p and a point file, what is the 

distance from p to the nearest point in the point file, i) Given a 

point and ? line segment file, which line segment passes closest to 

the point, j) Given a density nap and a district file, wh>t are the 
volumes in each district (spatial integral cf density), k) Given a 
district, what is the centermost point (an inside point which is 
farthest from the boundary). 

Effects of positional accuracy on estimates derived from multiple data 


planes . All mapping, whether satellite derived or obtained by conven- 


-All- 


tional means involves global and local positioning errors. When prodjcts 
are derived from the analysis of several mapped phenomena, the error 
is cumulative. Investigations need to be undertaken to determine: 

a) The sensitivity of models to positioning errors in input data sets, 

b) The potential for the high resolution satellite data to improve 
spatial integration/mapping functions such as trend surface mapping and 

development of multi-stage sampling designs. 

Investigate the performance and capacity requirements for the large 
record size a nd special purpose processing required for imaging and 
geographic applications . Is there a need to change the basic structure 
of Data Bass Management Systems? Or, can we expect fundamentally dif- 
ferent approaches to satellite data use in the future, such as sub- 
scene sampling for trend analysis rather than large area inventories? 
What v/ould be the impact on data administration and use for expected 
scenarios of satellite data use determined by selected case studies. 
Examination of possible and probable future environments which will 
impact renevjable resource data base marace-ent and use . Factors such 
as timeliness and repetitiveness of satellite overflights will con- 
tinue to be essential drivers of satellite systems. All stages in 
ground data acquisition, calibration, ceorcding, archiving, trar.sfer- 
ing, and analysis need to meet the satellite over-light requiremer.ts 
if an end product is to be provided. Giver, certain assumptions in 
future analysis requirements, v/hat are the key stages in data prepara- 
tion which need to have improved through-put? How would each stage 
in data preparation and analysis best be served (i.e., centralized 
or decentralized function)? Are basic structural changes implied if 
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a move from research demonstration to operational capability is to be 
achieved? 


Houston WorkshoD on Information Performance 
Summarizing the discussions held after the presentations during the Houston 
Workshop on Information Performance several important points were stressed. 
Professor Jensen made the point that if we want agencies such as the USDA's 
Statistical Reporting Service to change over and employ new ^•'nroHures baseo 
in some part on the analysis of remotely sensed data then we need to come to 
understand the sources and magnitudes of errors associated with all parts of 
the sampling systems to be used by such agencies. This point was echoed in 
subsequent conversations held with George Rosenfield of the U.S. Gt'ological 
Survey in Reston, Virginia (see correspondence Appendix D). In addition, 

Gaylord Warden stressed that while the office of Federal Statistical Standards 
and Policy (OFSSP) criteria are app*'opriate to natural resource statistical 
systems, neither the institutions which collect such information nor the concepts 


employed in the sampling systems are sufficiently developed to permit de- 
tailed tradeoff and cost benefit analysis wh'cn could be considered in any 
v/ay comparable to those undertaken for econc^'c and other Federal statistical 
statistical systems. 

A number of participants discussed the '•=ed for an ir.oro^ed method 
for getting at the question of user data requi r=-.ents . Co"r.ents such as 
Clough's: A basic examination of appr.'.p ..i"? ret.hads for determining what 
agencies and institutions are trying to aciiieve must be f.c comp 1 i shed before 


any bona fide analysis can take place; . r, optimization cannot occur until 


we understand the nature of the mr.rket. O'dell ta''ked of the potential for 
the use of case studies and/or the value of a series of well documented 






data sets. These could be employed to test and improve statistical sampling 
strategies or to develop new strategies which are more robust and sensitive 
to environmental variations. Such new or improved strategies are necessary 
because for many renewable resource survey problems classical statistical 
techniques are "nearly applicable." O' dell employed the phrase "nearly 
applicable" because most of the assumptions upon which classical statistical 
procedures are based are violated in their application to specific resource 
inventory problems. This is important as while sampling is still a major concern 
a number of participants felt that methods for achieving improved verification 
f'l" local identification and classification in a non-sampling approach is of 
r.ore importance. 

There was also a considerable discussion of the nature of the accuracy 
M'lestion in remote sensing applications. Cobberly pointed out that there 
‘T»'o many levels of accuracy assessment needed. We need to study the accuracy 
V '. infonnation questions, as well as an analysis of accuracy vs. informa- 
tio-i, content vs. decision-making. The whole question of original data 
P'vci-jcts and how this fits into a total systems accuracy concept in remote 
^“'rsing must also be addressed. In addition, it was felt that processing 
‘■^‘'curacy must be addressed in terms of levels required within given time 
t'"'straints imposed by a decision system. This led to a ccrr.-^r.z by Houston 
t-’v’t four major factors seemed to be recurring in the discussion which nad 
‘^^'“•'siderable relevance to performance and required more st^cy. These 
^•’ctors are accuracy, timeliness, applicability and affordability. 

Finally, a number of study areas requiring fundamental research were 
P.:t forward based on these discussions. These would include studies on 
t^'c: 

• conte; of future uses of remote sensing systems, 
including the needs for truly global resources infor- 
mation systems; 
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e need for detailed censuses of resources vs. samplinci 
of given resources should be assessed; 

t development of concepts to facilitate tradeoffs and cost 
benefit analysis of economic and other federal statis- 
tical systems. These studies should address questions 
of privacy, costs, and accuracy verification; 

. • use of remote sensing to improve the accuracy of local 
statistics ; 

• new approaches to sampling more respc;;s.ive to variations 
from natural (normal) conditions; 

• resolutions necessary for the oroduction of key indica- 
tions of important resource conditions. 

Here again the context of future uses of remote sensing including the 
needs for global information systems can be viewed from a scenario or case 
study approach. Research here should address the range of socio-economic 
and geopolitical issues V thin which future global resources information 
systems will operate. Remote sensor systems of the future can serve a 
variety of user groups. The combination of artificial intelligence for on- 
board und ground processing and analysis of the data/information produced 
by these systems for input to potential global reso-.-rce plarning and manage- 
ment issues offers an exciting and potentially rewarding area for future 
study. 

To date most large scale resource applications of remotely sensed data 
have involved sampling in some form. Yet, resource managers typically re- 
quire maps as inputs in their decision process. This question of techniques 
and methodologies for providing a detailed census in cartographic fo»'m for a given 
resource should be examined. Issues v/hich must be addressed include scales, 
geometric corrections registration, projections, transformations, classifi- 
cation procedures, and classi ficdtion and locational accuracy verification 


-A15- 


OF iPGc/i» i 


procedures. In this area what must be looked at carefully is the potential 
for achieving mature mensuration status for future satellite remote sensor 
systems . 

The need for studies directed towards the development of concepts to 

facilitate tradeoff and cost benefit studies of economic and other federal 
statistical systems could potentially be accomplished as part of the studies 

aimed to develop a general framework for assessing the value of an infor- 
mation system which v;as discussed earlier. Included in these studies should • 
be questions of privacy, costs, public good, accuracy verification, utility 
factors, and tha development of tradeo'f options for obtaining a "minimally 
sufficient statistic." 

In crop estimation and forecasting of acreages and yields often remote 
sensing data are regarded as merely another source of information that can 
be used to improve the precision of current methods, if it can be shown 
that it would be useful for the purpose. Statistically, ordinary local sample 
data can be made more effective for estir.ation if auxiliary data can be found 
that is functionally related to the primary data and is avail =ble over the 
whole universe of interest. In the simplest case, if the directly obtained 
("ground truth") data “and the auxiliary (rer.cte-sensed) dati are correlated 
(either positively or negatively) then the auxiliary data are regarded as 
statistically useful for this purpose. If the remote-sensed data are 
available at no cost then it would be irrational statistically, to reject the 
use of those data in crop estimation, for those crops processing non-zero 
correlations between the two kinds of observations. Hence apart from tech- 
nical details and costs, the determination of these correlation coefficients, 
the conditions of their measurement, their stability over time, etc., are 
required before remote-sensed data can be expected to be accepted for crop 



estimation. If the correlations exist, then the intercept and slope coef- 
ficients of regression, a and B, are of interest and how stable they are over 
species, districts, time, etc. If we presume that the values of these 

coefficients are peculiar to each of these situations and therefore, for 
suitable accuracy, must be determined for each estimating situation. If 

they were constant then, once determined, the primary data ("ground truth") 
can be dispensed with. Since p, a and B are also dependent on the inter- 
pretation ("classification") of remote-sensed data and the size and type of 
ground truth unit used, such interdependencies suggest that a set of coef- 
ficients, and the nature of their dependencies on factors affecting them, 
would be very useful for evaluating the effectiveness of remote-sensed data 
for crop estimation. Perhaps such information exists now in some form. 
However, if uses are to be made in any part, or in every part, of the world 
a systematic collection of these "facts" would be very helpful indeed. 

Many sampling approaches employed in the analysis of remotely sensed 
data of renewable resources are based on concepts which involve the calcula- 
tion of means from records acquired over a given tire frame. Yet in many 
instances it is the short-term episodic events, whose true impact may be 
masked in normative data, which can produce disastrous ccnsequences for re- 
sources. Research into the best ways to account for these occurrences must 
be conducted if v/e are to design sensor systems capable of providing early 
indications of impending problems so that effective remedial management action 
can be taken. 

Somewhat related to this is the question of the best methodology for 
determining the optimum resolutions necessary to produce key indications 
of important resource conditions. Resolution is an important concept in 
remote sensing; current methods of testing the information which can he 
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extracted from remotely sensed data of varying resolutions leave much to 

be desired. Methods must be developed wherein optimal tradeoffs between 

information extraction potential and varying degrees of resolution are 
clearly layed out. We must know that when we wartt to survey the forest 

that we may not need to see the limbs on the trees. Yet, v/e must con- 
stantly be aware of the question as to whether multiple uses demand the 
determination of a minimally '••jfficient resolution and account for the 
tradeoffs involved in producing this data. 

After briefly summarizing from all v/orking groups the more specific 
research areas which should be addressed in the areas of data and informa- 
tion performance along with data and information attributes, the panel 
feels the following research topics could profitably be pursued: 

DATA AND INFORMATION SYSTEMS PERFOR-MANCE 

- Examine Fundamental Aspects of the Accuracy of Prc-ucts 
of Remote Sensing 

- Develop Relevant Measures which Characterize Cata/Infor- 
mation (e.g.. Minimally Sufficient Statistic) 

- Develop Procedures Necessary for Renewable Resources 
Satellite Sensor Systems to Achieve "Mensuration Systems" 

Status for Given Applications 

- Continued Examination of Fnndanental Ascects of Samel ing 
Theory as they Apply to Spatially Dist-ibuted, Temporally 
Varying, Renewable Resources Parameters; at Local, 

Regional, National, and International Scales. Key Issues 
Incl ude: 

• greater sensitivity to deviations from 
"normal conditions" 

• identification of key parameters and deter- 
mination of correlation coefficients between 
remote sensor scene derived data and ground 
conditions for use when no independent source 
of verification exists 

• explore the potential of ncnparametric test 
of data 


( 
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DATA AND INFORMATION ATTRIBUTES 

- Develop Irrproved Techniques for Measuring Tradeoffs Between 
Use of Existing and New Inforr:ation Systems for Renewable 
Resource Decision-Making 

- Investigate Impacts of Timeliness, Reliability, Accuracy, 
and Assured Product Delivery on Cost Benefit Potential 

- Examine Alternative Means of Assessing the Public Good 
and Multiple Use Aspects of Data/Information 

“ Explore Use of Existing Collateral Data to Reduce Re- 
quirements for Sensor and Ground Truth Data Acquisition 


In conclusion, an important point which has reoccurred in our dis- 
cussion again v/as raised by non-working g:'c ip \;orkshop pa»'tici pants: 
there is a lack of a theoretical framework for evaluating the products 
of remote sensing. This question must be addressed. As Clough pointed 
out, an analysis of this research area must address a variety of issues 


including credibility, public good, policies and standards, real time and 
archival temporal requirements, uniformity end starda^'dizaticn, data inte- 
gration and more. 

It is a big task. It is a task, however, which participants felt must 


be strongly addressed if we are to truly begin to realize the irfornation 
potential inherent in .remotely sensed data. 
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The material contained here is an outgrowth of a series of studies to 
develop a program of fundamental research which is being sponsored by the 
Renewable Resources Branch of the Earth and Planetary Explo»*ation Division 
of NASA's Office of Space Science and Applications (OSSA). The purpose of 
these studies is to define a three-to-five year research program to broaden 
and strengthen the scientific base for future applied research and development 
programs directed at better use of aerospace and remote sensing in monitoring 
the Earth's renewable resources. 

The future potential of aerospace remote sensor systems is truly ex- 
traordinary. The opportunities they provide, however, will require public 
decisions concerning satellite sensors, ground facilities, and analysis 
systems of great complexity and cost. The potential benefits of future sys- 
tems are large, but similarly large are the data management and storage 
problems they generate. Various systems could provide information to address 
important national and global resourcf planning and management concerns, but 
they would also create legal, social, and economic problems of national and 
international significance. 

OSSA currently has a number of applied research and development projects 
(e.g., AgRISTARS) which employ essentially existing knowledge to design, 
engineer and demonstrate the capabilities of aerospace remote sensing as a 
valuable resource management information source. The basic research program 
is intended to compliment these project" by focusing on concepts and issues 
at the frontiers of the relevant sciences. A successful program will pro- 
vide the essential building blocks for future applied projects to support the 
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design of sensors, ground facilities, and institutions to facilitate national 
and possibly increased international use of remote sensing systems in the 
late 1980's and 1990's. The research will be conducted by scientists who 
are knowledgeable concerning current and potential applications of aero- 
space remote sensing technology to renewable resources. Research results are 
expected to be communicated worldwide by the publication of papers in recog- 
nized scientific journals and other types of appropriate publications. 

The Program Definition Study of Basic Research Requirements in remote 
sensing is divided into four general areas of study: 

* I 

(1) Scene Radiation and Atmospheric Effects Characterization; 

(2) Pattern Recognition and Image Analysis; 

(3) Electromagnetic Measurements and Data Handling; and, 

(4) Information Utilization and Evaluation. 

This document represents the final report of the working group on information 
utilization and evaluation. Each of the four research area working groups 
were composed of scientists ^nd practitioners from universities, research 
institutions, industry and government. These groups were formed to plan 
and conduct a series of workshops to identify critical research topics and 
to disseminate knowledge of current and projected applied research and 
development needs. Each working group is developing a prioritized list of 
research topics and a preferred method for implementing and coordinating its 
recommended program of basic research. Each group leader or leaders is or 
has been supported by selected members of the working group who are respon- 
sible for documenting the group's findings and recomnendations in a report 


to NASA. 
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The st'-idy undertaken by NASA is based on the premise that the develop- 
ment of a successful basic research program requires careful planning and 
communications. Planning and the ultimate success of the research effort 
can be enhanced by extensive use of scientists and practitioners with dif- 
ferent interests who are familiar with potential applications and problems 
witn information based on remotely sensed data. Communication between NASA 
pejsonnel and the research and user communities is more likely to identify 
worthwhile research topics and stimulate the scientific comnunity to engage 
in such research than a purely internal effort by the agency. Therefore, 
NASA asked working groups to conduct a series of workshops where qualified 
experts from universities, research institutions, private industry, and 
government hear and discuss one another's views concerning the current state 
of knowledge, on-going work at the frontiers, and critical areas where 
research could produce significant results. The workshops have and are pro- 
viding opportunities for working groups to draw broadly on the knowledge of 
the scientific community and deepen their understanding of relevant issues 
before preparing recommendations to NASA. 

The working group on Information Utilization and Evaluation organized 
and then met in organizational and planning sessions in April, 1980 in 
Santa Barbara, California. Members of the Working Group Steering Committee 
are seen in Table I. 

TABLE I 

INFORMATION UTILIZATION AND EVALUATION WORKING GROUP 
STEERING COMMITTEE MEMBERS 

GLEN BACON, IBM Corporation 

NEVIN BRYANT, NASA Jet Propulsion Laboratory 


— - 
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CHRISTOPHER CLAYTON, University of California 

LUDWIG EISGRUBER, Oregon State University 

JOHN ESTES, University of California 

FORREST HALL, NASA Johnson Space Center 

IDA HOOS, Space Science Laboratory, University of California 

ROBERT MACDONALD, NASA Johnson Space Center 

BRUCE SCHEER, The Planning Economic Group, Boston 

RONALD SHELTON, Michigan State University 

CHARLES VARS, Oregon State University 

A subsequent meeting was then held in July 1980 at Johnson Space Center 
to: (1) identify those areas where basic research is appropriate and (2) 

plan a series of workshops to address these research needs. A listing of 
participants for each infonnation utilization and evaluation meeting and work- 
shop can be found in tabular form in Appendix E of this report. 

Following these initial meetings of the Information Utilization and 
Evaluation Working Group a series of three major workshops and a special 
session (held in conjunction with a National Meeting on Remote Sensing for 
Resource Management) were held. The first session was conducted in September 
1980 on Information and Decision Processes at Asilomar, California, and a 
special session in Kansas City in October 1980; the second workshop on Data 
Base Use and Management in San Jose, California in January 1981; and, the 
third workshop on Data and Information Performance in Houston, Texas in 
February 1981. 

In February 1981, just prior to the final workshop, a third planning 
session was held in Houston to discuss progress that had occurred to date 
and to plan for the final workshop along with how this final report would 
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be assembled. As previously stated, participants at these meetings are 
listed in Appendix E of this report. Table II, however, presents a break 
down of the individuals attending each of the meetings at the Information 
Utilization and Evaluation working group by employment. 


TABLE II 

BREAKDOWN OP TOTAL PARTICIPANTS 
PLl PL2 AW KC SOW 


PRIVATE INDUSTRY 

FEDERAL GOVERNMENT 

STATE AND LOCAL 
GOVERNMENT 

UNIVERSITY 

TOTALS 


8 

10 


5 

10 


2 

3 

3 

S 

16 


3 
2 
1 

4 
10 


3 

17 


BY EMPLOYMENT 
PL3 HW OTHER 

2 

2 5 1 

4 7 4 

6 12 7 


TOTALS 

16 

25 

4 

43 

88 


Since certain individuals may have attended more than one meeting a breakdown 
of individual participants by employrr.ent is also seen in Table III. 

TABLE III 

BREAKDOWN OF INDIVIDUAL PARTICIPANTS BY EMPLOYMENT 


PRIVATE INDUSTRY 14 
FEDERAL GOVERNMENT 16 
STATE AND LOCAL GOVERNMENT 4 
UNIVERSITY 22 
TOTAL INDIVIDUALS PARTICIPATING 56 


These tables illustrate that while overall state and local governmental 
representation in the working group was low, participation by University, 


private industry, and federal government representatives was relatively 


evenly split. 
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The Initial planning meetings were basically a time to exchange Ideas 
and formulate a perception of the task ahead. Me;..bers of the Working Groups 
Steering Committee found In their discussions that although there Is no 
single theory of Information, they could come to some agreement on a general 
framework encompassing the major features of Information systems. The major 
elements the framework, as well as a paradigm within which to evaluate 
Information systems problems, are sketched here to provide a understanding 
and guide to the concerns which underlie the recommendations for research 
p<^esented later In this report. 

At their most basic level Information systems are designed to help 
decision-makers make decisions toward solving problems. This Indicates why 
theoretical c»* conceptual work on Information should focus to some extent 
on problem-solving. Since problem-solving requires some understanding of 
the nature of the problem to be solved, information can be viewed as the 
product of some process of Inquiry. From this perspective, any knowledge 
about the problem Is dependent on the system of Inquiry used to obtain that 
knowledge. However, to contribute to solving problems, an Information sys- 
tem must represent the •.-eallty relevant to those problems and the decisions 
to be made. As a consequence, the goals and values of the decision-maker 
affect the design of an Information system, and the decision-maker must be 
considered as a major component of the information system. 

The problems of Information system design and evaluation are relatively 
simple If the decision-maker confronts well -structured problems, but very 
troublesome If problems are Ill-structured. Problems are wel 1 -structured 
when known relationships exist between actions, outcomes, and their value 
to the decision-maker. The mode of Inquiry Is straightforward If problems 
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are well -structured, and the evaluation of information systems addressing 
such problems pose few difficulties. 

In contrast, ill-structured problems are those where knowledge of 
states of "nature," acts, outcomes, and even the values of the decision- 
maker are not known with confidence. The uncertainty inherent in ill- 
structured problems often mean there is difficulty in defining: (1) the 

reality and problems that information systems must address; and, (2) the 
scope of information systems themselves. Owing to these problems of defi- 
nition and scope it is our opinion that a multidisciplinary systems approach 
to problem-solving can provide the surest foundation for the information 
systems paradigm presented here. As discussed in this report information 
systems are broadly viewed as integrating the results of many different 
modes of inquiry to solve problems. Decision-makers apply heuristic as well 
as algorithmic approaches tc the acquisition of information and decisions. 

At this point it is important to note that data is often incorrectly 
equated to information. The paradigm outlined below: (1) distinguishes 

between data and information; and, (2) relates data collection to the in- 
quiry and analytical processes used to solve problems and to reach decisions. 
A data system seeks to represent reality empirically. Reality is complex, 
and data systems commonly report on multiple dimensions of the following 
fields of observation: (a) internal operating environment; (b) external 

operating environment; (c) external peripheral environment; and, (d) self- 
representation. Data can be Quantitative or qualitative. 

Measurement occurs only after deciding what to measure. Superior sampl- 
ing procedures and measurement techniques have little value unless the data 
generated are based on meaningful concepts that are relevant to the decisions 
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being made. Therefore, data collection involves three steps: (1) conceptuali- 

zation; (2) operationalization of the concept; and, (3) measurement. 

Bonnen (1977) emphasizes that the statistical reliability of a data 
system has :hree corresponding meanings. The first is conceptual reliability, 
(i.e., do the concepts accurately represent reality and pertain to the 
decisions being made). The second is operational reality (i.e., the cate- 
gories of empirical variables should be highly correlated with the conceptual 
representations of reality). The third is measurement reliability as usually 
defined by statisticians. 

Data systems produce data nnt information. To become information, data 
needs analysis and interpretation to acquire meaning for use in decision- 
making. As Riemenscheider and Bonnen (1979) observed, "an information system 
is a process which imposes form and gives meaning to data." An information 
system has three components: (1) a data system; (2) the analytical and 

interpretative capabilities to convert data into information; and, (3) 
the decision-maker. This is depicted in Figure 2.. 

Information systems have multiple components because decision-makers 
seek different types of information on the various "fields of reality" with 
which they must deal. Conceptual obsolescence is a problem for information 
systems because changes occur in reality and the agenda for decisions. The 
memory of an information system is invariably more appropriate in part to 
some prior reality and agenda than it is to the present. Modification of 
information systems attempts to overcome recurring mismatches of reality, 
decision, and available information. 

Communications within and between organizations require interpersonal 
transmission of data and information. Problems of codes, formats, and "noise" 


Figure 1. 


An information systems paradigm. 



Reality; Fields of observation: 

a. Internal operating environment 

b. External operating environment 

c. External peripheral environment 

d. Self representation 


Figure 2. Two modes of information use. 
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are widely understood, but those deriving from different conceptions of 
reality, problems, and needs are not so \/ell understood; although they can 
significantly affect the use and value of information systems. 

Information systems are used in both an ordinary and an extraordinary 
mode. The first mode occurs when information accords reasonably well with 
expectations and only routine adaptations are called for within the operat- 
ing environment. In ordinary circumstances recurrent mismatches between 
observations and expectations do not occur, and there is no need to change 
the information system. Path I in Figure 3 depicts the ordinary mode of 
information use. 

In the extraordinary mode, however, information does not accord with 
expectations concerning the operating or peripheral environments or itself. 
Non-routine actions are taken and learning, development, and innovation are 
then initiated to transform the information system. Components of the 
system are modified, deleted, or added in the extraordinary mode of use. 
Path II in Figure 2 illustrates the extraordinary use of information to 
alter the information system itself. 

The distinction between the ordinary and extraordinary use of informa- 
tion is significant but difficult to draw precisely. For situations exist 
where both modes are carried on simultaneously, and learning can occur when 
an information system ope-ates in the ordinary mode. Further, some systems 
may have limited capacity to review their own performance and to decide how 
to modify themselves. 

Based then upon our general agreement the Working Group's Steering 
Committee put forth four specific areas of critical importance to the area 
of information utilization and evaluation as we understand it. These are 
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four areas where we feel fundamental research must be conducted 1f we are 
to truly advance the applications potential of future satellite remote 
sensor systems. These areas identified and addressed in the discussions 
are as follows: 

(1) Information and decision processes; 

(2) Data and information attributes; 

(3) Data base management and use; and, 

(4) Data and information systems performance. 

Briefly expanding upon the core of the rational for the choice of 
these areas the Working Group Steering Committee members presented at the 
planning meetings their feelings in respect to: 

(1 ) Information and Decision Processes 
Present resource decision systems and their actual potential 
use of remotely sensed data are not well understood. Funda- 
mental research must focus not only on the physical design and 
image acquisition characteristics of perspective remote sensor 
systems but on current and probable future institutional arrange- 
ments, decision processes, and their information requirements if 
rational choices are to be made among potentially competing >'e- 
mote sensing systems. 

Information components of decision systems must be reviewed 
to specify and understand the characteristics, credibility, and 
utility of currently used data from all sources . It is not 
feasible to analyze all systems where remotely sensed data are 
currently being or potentially could be used. Therefore, the 
Working Group feels that research should focus on a limited 


number of application scenarios where remotely sensed data 
are or may be combined with data collected by other means 
for use in the decision process. 

Research efforts in this area should address both 
private and government decisions with national and possibly 
international implications. Utilization of biological and 
social science models in the decision process should also be 
analyzed. Actual practice in the use of scientific models and 
data could be compared against best possible practice given 
the current state of knowledge. These comparisons may then 
be employed to establish the quality of data and information 
currently used in decision-making. Such comparisons could 
also idenfity where modified information systems and scientific 
models that rely on remotely sensed data could be employed 
to specify the potential benefits which might accrue from im- 
proved sensor system capabilities and how those advances could 
improve decision processes in the next 10-20 years. Potential 
improvements would be judged by criteria appropriate to evaluat- 
ing the various consequences of monitoring the application 
scenarios chosen on a local, state, regional, national, and/or 
international basis. 

(2) Data and Information Attributes - 

Users of remotely sensed data typically utilize data and 
information with a wide variety of attributes. Attributes such 
as function, scale, timeliness, precision, and others should be 
examined in the context of systems where tradeoffs among attri- 
butes and institutional constraints can be directly addressed. 


Functional requirements of remotely sensed data are important 
and the desired scale, spatial resolution, and other attributes 
of the desired information will determine the resolution re- 
quirements of future sensors. Will system users want to see a 
forest, a timber circle, a stand of timber, or individual trees 
How will these elements be related to data needs? How will 
remotely sensed data interface with data from other sources? 
Fundamental research is needed to understand how the attributes 
of data and information derived from it interact and could 
affect decision processes in the future. Therefore, the 
attributes of data and information will be examined and 
evaluated within expected future decision environments con- 
cerned with cereal grain production, land use/cover, and 
soil moisture. Studies will address the wide range of data 

potentially produced by remote sensing technology. 

( 3 ) Data Base Management and Use - 

Many resource managers and scientists believe that in the 
future various renewable resource management questions will 
require new forms of datj base management (storage, manipula- 
tion, and retrieval) and uses (data outputs) to function ef- 
ficiently. Research in this area will analyze the basic 
factors that affect: 

(a) Input of remotely sensed and other data into infor- 
mation systems; 

(b) Storage, retrieval, and management issues; 
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(c) The interaction and transmission of remotely sensed 
data and other data once they have been entered into 
a system; and, 

(d) The types of products demanded of information systems. 
With respect to the first and second areas, research will 

seek answers to basic questions concerning data types, quantities, 
formats, storage, accessibility, retention spans, and accuracy. 

In the third area, research will address how resource management 
data systems can be merged, manipulated, analyzed, and inter- 
preted once remotely sensed data is incorporated into the system. 
What are the models employed, the types of data the require, how 
can these data be combined, surveyed, and sampled? The final 
area involves examination of the factors affecting the presenta- 
tion of information to its users. Topics will include hard vs. 
soft copy, tabular, graphic, carto-graphic, and combination 
products, as well as analyst-decision-maker interactions. 

(4) Data and Information Systems Performance - 

This general area is concerned with investigating approaches 
to performance evaluation of the data and information generated 
by remote sensing mapping and inventory systems. The history of 
remote sensing reveals recurring problems of (a) defining per- 
formance parameters meaningfully to both technology users and 
developers, and (b) establishing approaches for estimation of 
those parameters in constrained situations where independent 
reference data may not be available. 
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To establish the meaningful performance parameters both 
technology developer and user interests must be taken into ac- 
count. Whe eas the developer may be primarily concerned with 
the accuracy of the estimates of the subcomponents of the 
estimation models, the user may be more generally concerned 
with questions revolving around the timeliness, utility, and 
reliability of the estimates, their cost, and interpretability 
and familiarity of the products of the technology, along with 
ultimately understanding the reports, etc. Additionally, 
the technology developer may be concerned primarily with quan- 
titative measures of performance, while the user will be con- 
cerned with more qualitative social and economic factors. 

It is difficult to estimate the impact of oerformance 
parameters once they have been established. In most cases 
remote sensing mapping and inventory systems generate data 
and information which are independently available from in- 
place conventional systems. In all cases, the data and infor- 
mation from the remote sensing system are being evaluated for 
that improvement in either timeliness, spatial dimension or 
accuracy as compared to such conventional systems. This then 
raises the fundamental question of how the performance of the 
remote sensing system is to be evaluated (i.e., to what 
standard are its outputs to be compared?). How are infer- 
ences as to its perfomiance to be made in situations where 
independent data is either unreliable or non-existant? What 
should be the measures of performance? 
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The sections which follow present a brief synopsis of the individual 
presentations at the workshops which addressed these topic areas. Tnis is 
followed again by a summary and conclusion sections and recommendations. 
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The following material presented in chronological order provides an 
overview of the important points presented by speakers at the three informa- 
tion utilization and evaluation workshops. In addition also presented are 
the salient points acquired by steering committee members in special sessions 
and discussions held w«th participants at Kansas City in conjunction with 
the Soil Conservation Society of Americas meeting on Remote Sensing for 
Resource Management. 


Asilomar Workshop 

The first workshop conducted by the Working Group on Information 
Utilization and Evaluation was held at Asilomar, the University of Cal'I^'^rnia 
Conference Center at Monterey, September 16-17, 1980. Attendees are listed 
in Appendix A while the topics and discussants are seen in Table WR-I : 

TABLE WR-I 

Titles of Presentations and Speakers at 

INFORMATION AND DECISION PROCESSES WORKSHOP 
ASILC.-1AR. CALIFORNIA 
SEPTEMBER 1980 

THEORY OF INFORMATION SYSTEMS: PUBLIC AND PRIVATE DIMENSIONS - 

C. Bart McGuire 

IMPLICATIONS OF GLOBAL REMOTE SENSING SYSTEMS: A DISCUSSION - 
Ludwig Eisgruber, leader 

INFORMATION SYSTEMS USED IN PUBLIC AND PRIVATE DECISION MAKING - 
Bob Barker, Bob Potter, Leonard Sloski 

ARE THERE COMMON RESEARCH NEEDS?: A DISCUSSION - 

Charles Vars, leader 
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TABLE WR-I (continued) 

CRITICAL ANALYSIS OF EXISTING INFORMATION SYSTEMS - 
James Bonnen 

CULTURAL, LEGAL AND POLITICAL IMPLICATIONS OF GLOBAL INFORMATION SYSTEMS - 
Oswald Gan1e> 

CRITICAL ISSUES RESEARCH: A DISCUSSION - 

Ludwic Elsgruber, leader 

The worksliop fc:used on the topic, Information and Decision Processes . 

The objective was to identify those fundamental aspects of information and 
decision systems that are incompletely understood, important to future 
applications of aerospace remote sensing technology, and which required 
fundamental research in order to achieve application ready status. Parti- 
cipants sought to identify research topics that are common to many informa- 
tion systems and are on the frontiers of particular disciplines, or cross- 
cut the usual boundaries between disciplines. 

The Workshop opened with a presentation by Robert B. MacDonald con- 
cerning the goals and status of the NASA program definition study of basic 
research requirements. MacDonald was followed by John E. Estes who explained 
the purpose of the Workshop on Information and Decision Processes. Six 
major presentations were made by well-known authorities. Summaries of 
these presentations, identified by participants, are provided here. Critical 
Research areas identified by participants can be found in Section V of 
this report. 

Theory of Information Systems: Private and Public Dimensions 

C. Bart McGuire, Graduate School of Public Policy, University 
of California, Berkeley. 


McGuire reviewed the nature ol benefit-cost analysis, its successes and 
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failurss, and ths applicability of its n^tnoJs to t !»2 ovaUiatiOii of infomation 
systeus. Hs argued tiiat decision analysis provides an appropriate theoretical 
framework for addressing many questions about irjfor.ration systems; its prac- 
tical impact, however, has been limited because the model of rational choice 
breaks down where there are two or more persons, each trying to achieve their 
own Objectives. Although considerable recent work has focused on these 
problems, no consensus has yet emerged on tiie "correct" way to look at group 
decision-making problems. 

McGuire also discussed the problems involved in evaluating public 
inforiiiation systems. Problems for public systems are greater than for 
private systems because publicly provided information has special charac- 
teristics and impacts on private and public decisions. Evaluations of public 
systerrs must answer questions about how to diseminate and finance, the 
avoidance o: politicization, errors, and tn? creution ui false expectations, 
as v/all as the basic question of what to provide for whom. McGuire concluded that 
cltnougii some good ;/ork has addressed these questions, it does not provide 
an adequate basis for confident evaluations of existing or proposed public 
information systems. 

Information Sources Used in Private and Public Decision-Makin g 

1. G. Robinson Barker, i-lanagsr. Forest P.esource Information Systems, 

St. Regis Paper Company. 

Barker described how management decision-mar.ing by St. Regis Paper 

Company depends on fore.'t inventory data acquired from remote 
sensing. He descrioed the problems as well as the benefits of obtaining 

and maintaining accurate inventory estimates. St. Regis nov/ uses high 
flight aerial photograpns and Landsat data in combination with traditional 
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r. 2 thods to acquire forest inventory data. Aerial photography and Landsat 
products are employed to design timber cruise strategies, annotate previously 
acquired inventory and cartographic materials, and develop documents 
for work in tne field. Remotely sensed data are very useful , but high 
acquisition and storage expense, rapid obsolesence timely availability, 
vendor reliability, and sample design remain important problems. 

Barker empnasized that Landsat data are ancillary data for St. Regis, 
naps are the desired primary data. The digitizing and interactive graphics 
capaoilities tnat St. Regis has developed to produce maps v/ith Landsat 
data generate significant benefits to the company. Landsat products permit 
St. Regis to sample more efficiently, reduce design costs for aerial sa.mp- 
'iir.-, anc improve its aoilicy to monitor forest resources Important to t.ie 
company. Landsat products facilitate the dissemination of information 
and almost eli.minate the need to draft naps by conventional means. Barker 
concluded by expressing his concern whether in the future Landsat data will 
be (1) continuously available in a timely fashion and (2) compatible with 
existing data. 


2. Roberc Rotter, Assistant Chief of Planning, California Department of 
Resources. 

Rotter began by describing the activities of the California Department 

o' hater Resources. The Department engages in planning activities that 
affect tne California '.-.'ater and Central Valley Projects as well as the 

use of water throughout California. The Department designed, constructed, 
and operates the Cul ifornia Water Project which transfers water from 
nortnern to southern California. Persuasion and litigation are important 
activities by which the Department seeks to influence the operation 
of the Central Valley Project as well as the use of water throughout 
Cal i fornia . 
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Potier explained the various types of data the Departnent uses in 
its activities. These include data on land use, water run-off 'and 
stora'je (surface and underground), snowpack, soil noislure, land cover, 
and vieather conditions. The data a>^c used with various models to 
estimate existing, and predict future, magnitudes of floods, 
v/3ter supplies, and v/ate>^ use. The Department also uses the data to identify 
seepage and plant stress, control encroachment , and conduct various 
conservation programs. 

Potter indicated that the models utilized by the Department are 
generally inferior to the available data. (The exception would be the 
flood prediction models which are superior to the data now available.) 

The recent drought put great pressure on the Department to generate 
estimates of the costs and benefits involved in various potential 
incremental reallocations of water. The Department seeks to avoid the 
politicalization of its data; this is difficult, however, because 
pre- and post-project assessments by the Department sometimes lead 
to actions with significant economic and political consequences. 


Les'srd Slosky, Assistant to the Governor of Colorado for Science 
enc 'econo logy. 


Feeer 
respo 
pi an 




Slcsvv first discussed the needs of state and local government for 
= 1 r= 50 ..rce and environmental data. He stressed that both state and 
al legislation and regulations have increased the authority and 
nsiciiity of state and local governments and regional agencies to 
anc “a'.age natural resources. To execute these responsibilities 
..•ithin their I'-.itet budgets, state arid local officials 
spatially orieiited data which are comprehensive, timely, and can be 
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■ reasonable cost on a repetitive basis. Coventional data 


coUrCtic' tech.niques are often costly or incapable of nesting these 


Slos'.y reported that the Intergovernnetital Science, Engineering, 


and 7e:r. .o'.D:y Advisory Panel (ISETAP) Natural Resource and Environment 


’asL Force fcj-.i Landsat tc be =n important technology that can make 


sigr.ificar.- , often unique contri outions to the information bass required 


for state aoc local government's management of natural resources. Surveys 


cono.ctec ov tne Task Force indicated that state and local governments find 


Landsat to be valuable because: 


• Landsat can provide the only feasible means for conducting 


certain natural resource inventories. The cost, time and man- 


:ci-.;er involved in the use of conventional techniques is often 


Or^-'iiDUlVe. 


• Toe -eoetitive coverage of Landsat can permit users to no-iitor 


•'■etjently the cnanges occurring over time in large areas. 


• me mul ti -spectral aiivi synoptic characteristics of Landsat can 


orcvide a nev means of viewing the environment , providing information 


virtually impossiole to acquire v/ith conventional techniques. 


Landsat can provide uniform standardized data in a digital 


for'at whicn is easily incorporated into computerized information 


i i i . 


Slos.-;,' reviev.ed toe actual use of Landsat by state, regional and local 


egenpies, an: tnen outlined the constraints on its increased utilization, 


T.oese constraints include: 
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• LdC<: of Federal CoT:n tment to Data Continuity and Compatibility . 

Un;:'!! ar. ori='*3tio'‘ia1 Landsat In for '.ation Systain is “Stabl ish 2 d , 
staias v/ill r.asitate to invest ir. tiie developnant of Landsat 
capabilities and will delay the incorooration of Landsat data 
into ranage.'r.ent programs and deci sion-naLi-in . 

• u ac3 Timeliness : Delays and uncertainty in obtaining data from 

the ground system have and will oreclude the use of Landsat by state 
and local governments. 

• Inadequate Federal Technology 'iransfer : State, regional and local 

agencies do not have suffic'L-irc technical capacity or specific expertise 
in '-emte sensing and digital analysis techniques' They need more 
tecr.riical assistance to j£ -elop an initial capability and continuing 
assistance to stay abreast of technological developments. 

• 111 -defined Federal A gency Responsibilities . Lack of co- 
ordination among Federal agencies and ill -defined responsi- 
bilities for the Landsat system have led to limited services 
to state and local governments. 

• Failure of Federal Agencies to Use and Encourage Landsat Use . 

Federally mandated p-ograms impose great demands for new data. 

Hov.'ever, from time to time Federal agencies have not permitted 

states to use Fed^ially provided funds for Landsat processing 
to meet program requirements . 

• LacK of State Involvement in Landsat Deci sion-nakinq . The Landsat 
system has been planned and operated for the Federal and inter- 


national sectors winn little input from state and local govern- 
ments. Federal agencies have not recognized the operational 
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reqjirenants of stat 2 and local ag3nci3S or th= uniqj 3 technical 
and institutional setting of each government. 

• State Constraints t o U se of Landsat. The fragmentation of natural 
resource responsibilities and the attendant difficulty of coord- 
ination, the lack of trained staff, bureaucratic inertia, and 
difficulties in obtaining funding for nev/ programs have also 
constrained the use of Landsat. 

Critical Analysis of the U S. Federal Statistical System 

James T. Bonnen, Department of Agricultural Economics, Michigan 

State University, and erstwhile Executive Director, President's 

Reorganization Project for the Federal Statistical System. 

Bonnen reviewed the development, characteristics, and problems of the U.S. 
Federal Statistical System. Unlike most countries, the United States has 
a decentralized statistical system. For most of its history the system 
has worked well. Agencies have developed model statistical programs to 
serve policymakers and other users; they have also pioneered in the 
development of new technologies. 

In recent years, however, the system has lost some cf its effectiveness. 
Major new demands have been placed upon it and there have been significant 
changes in the environment ir; which it operates. Billions of dollars in 
the Federal budget are allocated to State and local governments under 
legislated formulas based on Federal statistical series. In addition, 
certain Federal expenditures impact private sector wages and are in turn 
affected by Federal statistics on prices. Government agencies require 
statistical data to support policy decisions and the evaluation of their 


activi ties . 
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Bor.'-.2n i'-Tj2d that national policy infomation needs now require a 
statistital syste" with strong overall coordination and planning. Without 
such dirrcti^n, a decentralized system cannot deal effectively with the 
proclsMS '.M' afflict the current fragmented, uncoordinated collection of 
stasisii::! activities. These problems include: 

• La:-: of policy re l evance . While the current system provides a 

.•/ide variety of data, tliese data are not employed adequately 
for objective analyses of complex issues which cut across 
present Department and policy decisions structure. The system 
has frequently failed to anticipate the need for data to be 
used in dealing with emerging policy problems. Failure to 
anticipate majer data '•equiraments leads tn unneccessary conflict, 
ove*" the nature of problems and policy options; it also leads 
to rising political pressures on statistics and statistical 
agencies. 

• threats to integrity . For tiie public to trust the data, 
it must be sure tnat the data are not subject to actual or 
perceived manipulation and that the objectivity of Federal 
statistical data is fully orotected. While the record is hvood, 
t.-.ere have been occasional problems. 

• Inade cua te quality . The statistics produced in small units, or 
as oy-prodbct, are often unreliable and poorly designed for tiieir 
purpose. The growing volume of contracted data collection and 
analysis are too frequently of poor statistical quality. 


f 
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• Inadecjate protection of privacy of respondents. 

• Excessive paosrwork . 

Bot'r er, concluded by exolaining why the Statistical Reorganization Project 

recG ’‘toricec , end tne Presideni approved, establishment of an Office of 

Stetisticil -olicy (OS?) in tne Executive Office of the President for the 
coorJinetion of c.he Federal Statistical System. This Office v/ould function 

as a separate eger.cy reporting to the President and accountable to the 


Congress . 


The missicr. of the Office would be to: 

• Ensure the policy relevance, quality, and integrity of statistical 
data and analyses produced by agencies of the Federal Statistical 


• De/elo: plans so that the Federal Statistical System will be able 
to "eea future information needs efficiently in tlie face of 
tec.-nol o;;ical , social and economic changes; 

• Assise in ninimizing the burden on all persons and organizations 
as'-.ei eo supply statistical or ocher data to the Federal Government; 

• .‘'.aincsir a proper balance bet’./ee.i protecting i.idi vidual and business 
ri*. 05 e: , *ivecy confi'Je..eis'i i ty a.'.i ’’eetinj i i^er-.ation neris 
TO- pjolic policy. 


Polioical, C.jle^ral, and Legal Problems of Global Information Systems 

Osv/ald Gar, ley. Executive Director, International and Allied Areas, 
narvard tri versity Program on Information Resources Policy, and 
srst\;hile Deputy Assistant Secretary of State for Science and 
Tecr.iolopy. 

Ganley discussed the political, cultural, and legal aspects of global 
remora 5 =nsi.-.r systems and their use. addressed three questions: 

1. ‘..'.nat will the political v/orld look like durim the next twentv 
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1. V.-M problsns ar 2 associatso wiui laryc', rop?titiv 2 , and loivj- 
cer.-n flov.’s of data across national borders? 

3. Lnat legal and political problems affect the international use 
of rerotely sensed data? 

Ganiev predicted that the v/orld during the next two decades will 
experience recurring political struggles. Uncertainty wil I characterize 
most rel3tic-:s between nations, regions, and groups, and leadership will 
be epi'ie.'eral ans limited, neither the United States nor any other nation 
will dominate or consistently provide leadership in international affairs. 

T!ie United States, despite its great pov/er and leverage, must negotiate 
witn otnar .nations and international organizations to achieve its objectives. 

Ganley too‘< the position that the United States governiient has just 
beg^n to urcerstand tnat the world has entered an "Information Age." 
Information is a basic resource like materials and energy. The need 
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ho wants it? How can you get il? and 


V/hat are the terms of trade?" links any resource to political or economic 
pov.er. For information resources, Ganley argued that traditional answers 
are now inadequate ov/ing to tlie growing abundance and versatility of 
modern technology. 

To illustrate the changing and uncertain answers to questions concerning 
information resources, Ganley drew on his previous analyses of tlte role of 
infC'*'!ation resources in Canada and of their influence on U.S. -Cana<lian 
relations. Tne possibility of rapid, massive, and cheap flow of data 
across Dorde-'S and tne preeminence of U.S. facilities and services has 
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■i;c -=ny countries to v/orry about transbordor data flows. Western European 
jov: CMts justify these concerns in tefTns of individual privacy. But 
tns Zinadian governrnent and industry view then as essentially economic - 
2 i-fiter of lost jobs, negative effrcts on the balance of payments, and 
7ore:on= nanagenent ooportunities. (See Figure WR-1). 

-.esearch and development in high-technology industries has also been 
= ■- ■'li-e. The Canadian government has expressed displeasure over 
tne '.z.i level of research and development performed by U.S. subsidiaries 
in C;-.aca and the substantial U.S. ov/nership of the Canadian electronics 
industry. These sentiments are analogous to those expressed by 
j.S. ir, Jjstries about Japan. 






"re Canadian governnent perceives Canada as being overwhelmed by 
■pan media content that robs Canada of its national identity. Directly, 
affects such U.S. industries as publishing and broadcasting. 
ii.'.ilar perceptions in the Third World pit restrictionist attitudes 
'St .-.nerican devotion to the principle that information . should flow 
ly. The resulting clashes have ideological as well as economic 


o/e'tenes. 

Tne possibility of broadcasting directly to homes via satellites raises 
si-i'ar issues. Canada wants to have domestic television broadcasts from 
s-'.'v'lites rather than local transmitters. But this could mean 
' ^'S'ca'ion of even more 1.5. -made content than at present. Globally, many 
--i cell eve that direct television broadcasting by satellite from on® 

C 3 - :'*/ to another witiipui the prior consent of the receiving state is a 
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violation of national sovereignty. The resulting conflict between the 
principles of free flow of information and of national sovereignty is 
fueled by the fear that the United States would use it great technological 
advantage for political, cultural, or commercial purposes. Ganley argued 
that the exploitation of Landsat satellites for agricultural and mineral 
resource mapping, already complicated by potential conflicts with intelli- 
gence interests and by questions of public versus private, monopolistic 
versus competitive ownership structures, is subject to similar fears in 
the international arena. 

Ganley concluded that enormous problems impact the international 
use of remotely sensed data because the concerns of nations differ greatly. 
He stressed the need for flexibility, both political and economic, in the 
design and operation of future systems to serve the international community. 

Kansas City Special Session 

Three members of the Information Utilization and Evaluation Working 
Group (Jack Estes, Nevin Bryant, and Charles Vars) conducted a special 
session at the Soil Conservation Society of America (SCSA) Conference, 

Remote Sensing for Resource Management , at Kansas City, MO, October 28, 

1980. The three-hour evening session brought together selected conference 
participants who would provide perspectives on fundamental research needs 
to supplement those obtained at the Asilomar workshop on Information and 
Decision Processes. Invited participants can be seen in Appendix A. 

Two individuals who were invited, Joe Cihlar and Bill MacFarland, 
were unable to attend the special session because of unexpected changes 
in their schedules, but Estes was able to meet with them in subsequent 
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interviews and their views are included herein. In addition to the 
syopsis of our discussions we have included in a copy of the results of 
a data needs question circulated at this meeting (Kansas City USER NEEDS 
SURVEY, Appendix C). These results were provided by Cihlar to Estes in 
December, 1980. 

Tile session opened with a presentation by Estes concerning the goals, 
status, and activities of the NASA Program Definition Study of Basic 
Research Requirements. Bryant and Vars then briefly reviewed the Workshop 
on Information and Decision Processes at Asilomar, as well as planned 
future activities of the Working Group on Information Utilization and 
Evaluation. 

One major research reconmendation emerged from the two-hour discussion 
session. This recommendation was that existing information systems, their 
use and users, and related processes need to be reviewed and documented 
before potential future systems can be evaluated. The objective should be 
the review and documentation of multi-purpose, government and private 
systems with international dimensions. The review and documentation 
process should not just include systems which obviously lend themselves to 
remote sensing applications. The option of combining remotely sensed data 
with data collected by other means should be studied. In fact, this latter 
possibility indicates why the review process should place special emphasis 
on why, how, and with what effects are multiple-source of data used within 
existing information systems. 

Most studies on the value of information assume, for example, that 
a farmer, an elevator operator, and a buyer all use the same information 
in the same way, but evidence indicates this is unrealistic. Some information 
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ls available as a public good, but it is not equally used by a11 who 

receive it. For these reasons, the identification of major market 
and nonmarket users of information, their sources of information, and 

the adequacy of these sources become important if new systems are to be 
satisfactorily evaluated. Particular attention should be given to the 
documentation of how the behavior of public and private users and producers 
of information are changed when information is viewed as a public rather 
than private good. 

San Jose Workshop 

The second workshop conducted by the Working Group on Information 
Utilization and Evaluation was held at Santa Teresa Laboratories of IBM, 
near San Jose, California, January 19-20, 1981. Participants are listed 
in Appendix A. The workshop focused on the topic. Data Base Use and 
Management . A list of individuals making presentations and the title of 
their talks are seen in Table WR-II. The objective was to review the state 
of the art in computerized data base managenent systems and technologies 
associated with the use of data-particularly remote sensing data. Eight 
presentations were made by individuals well versed in the field. 

TABLE WR-II 

Titles of Presentations and Speakers at 

DATA BASE USE AND MANAGEMENT WORKSHOP 
SAN JOSE, CALIFORNIA 
JANUARY 1981 

SPACE APPLICATIONS DATA SYSTEMS PROGRAM - 
Nevin Bryant for Peter Bracken 

PLANETARY SCIENCE DATA ACQUISITION - 
Adrian Hooke 
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TABLE WR-II (continued) 

REQUIREMENTS AND CHARACTERISTICS OF SCIENTIFIC DATA BASES- 
Stan Hansen 

CARTOGRAPHIC DATA BASES - 
Del mar Anderson 

COMMITTEE ON DATA MANAGEMENT AND COMPUTATION REPORT REVIEW - 
Ralph Bernstein 

DATA BASE ISSUES IN GEOGRAPHIC APPLICATIONS - 
Ray Lorrie 

PROBLEMS OF HIGH RATE COMPUTATION - 
William Sharpley 

DATA BASE CONSIDERATIONS FOR NAVY WEATHER FORECASTING - 
Eve Schwartz 

Summaries of these presentations are provided below. Critical Research 
issues from the discussion held at this workshop are found in Section V 
of this report. An appendix of selected viewgraphs given by each speaker 
is provided as background information (see Appendix B). 

1) P. Bracken/N. Bryant (presenting) 

NASA/OSTA Data Systems R&D Program 

The Data System has as its goal the development of key enabling 
technologies to assist the OSTA Discipline Branches more efficiently 
address the ground data handling of earth observation systems sensor data. 
The program cuts across all of the Discipline Branches, with the result 
that it has been structured to address both immediate and longer-range 
concerns to OSTA applications. Immediate concerns in improved data handling 
are being addressed through the development of Pilot Demonstrations and 
focusing on critical mission requirements such as the Landsat D ground data 
handling system. Longer range concerns are being addressed through the 


development of NASA expertise in information science and contributions to 
the data handling design for missions scheduled for FY86+ launch. Liason 
with OSTA research is .laintained, but emphasis again is on the Pilot Systems. 
The Branch's philosophy is to learn by doing, and build specific blocks of 
expertise. The Pilot Systems may in fact be too research scientist oriented 
for a complete evaluation of system requirements, given the stated OSTA and 
user community being composed of federal, state, end private organizations with 
high-volume and repetitive data processing characteristics. However, 
there are sore very challenging issues in extracting indexes and storing 
data associated with satellite infor-mation. These probably are not 
amenable to significant theoretical analysis given assumptions by high- 
volume and repetative data processing characteristics but are best 
handled on an experimental basis. 

2) A. Hooke 

Pl anetary Science Data Acquisition 

The planetary science data acquisition differs from the earth obser- 
vations program in two important respects: much lower data rates and a 

well defined and limited set of users. The process, however, suffers 
similar problems to that experienced by the earth observations missions. 

Being small, hov/ever, has enabled the program to recently redesign its 
information system from end-to-end, thereby highlighting elements of 
generic value to the Fundamental Research Program. Key elements in- 
clude: (1) exploring the value of data compression to trade off infor- 

mation for volure, (2) developing standardized data links between com- 
ponents for staging/transmission, (3) developing standards to permit 
expansion and external izaticn of the data network to more end users/ 


scientists. A principal enabling element has been the development of 
layers of protocols to keep track of packets of data being transmitted 
from spacecraft to ground and data base to data base. Layered proto- 
cols probably are the best way to completely inform the recipient of a 
data sets characteristics, but it is impractical to expect retroactive 
upgrading of protocol information in the initial data bases. 

3) S. Hansen 

Requirements and Cha racteristics o f Scientific Data Bases 

The Boeing computerized graphics and data base systems applied to 
airplane design, manufactur-e scheduling and logistical support was dis- 
cussed in the context of their technological requirements. Basic prob- 
lems in the Boeing environment are analogous to those in satellite 
information processing and utilization. Some key points brought up 
were: (1) we are not just computerizing data and procedures that were 

alrcady there, we are creating new approaches to analysing data and 
making comparisons through a greatly reduced cycle time for information 
processing. (2) Computerizing systems are vertically integrated to per- 
form specialized tasks with function-oriented software, while in fact 
what is needed is a global -oriented transfer capability so that trans- 
ferability of data can occur through horizontal integration. (3) 

Wnere many components characterize a system the number of horizontal 
interactions (e.g., facil i ty-to-facil ity) should be minimal and they 
should not be complex. (4) The costs associated with moving through a 
heterogeneous environment are associated with reformatting and converting 
data; roreover, imago and graphical data types and hardware do not have 
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co-plets corpatability of interfaces. (5) Compatibility cari be helped 
thro'jgh the development of a neutral format all components must conform 
tc, although this'does place an overhead on operations. (6) There needs 
to be decentralized data base structures, distributed data processing, 
and a cor.Ton carrier If the variety of users/ technology managers are to 
have a sufficiently flexible research and design environment to meet 
the va-'iety of custorrer/end-user needs/requests. (7) Users do not have 
requi regents, they have reactions to situations imposed upon them. 

4) D. Anderson 

Certocraphic Data Bases for Exploiting Remote Sensing Imagery 

It is irportant to note the difference between the cartographic data 
base r^r.ipulation and use for analysis versus that required for presen- 
tation. There is a need to provide simplified cartographic products 
to the end user v<hich get the salient points across. Hov;ever, this 
same data must be used for analysis as well as presentation. Therefore, 
you do rot want to have to change the data encoding structure over time, 
but rather have levels of generalization to handle more spatially de- 
tailed infomation. Furthermore, the cartographic data bases must be 
organized to accorrrnodate a wide variety of spatial attributes (points, 
lines, areas) and integrate them in a spatially consistent manner for 
analysis and simplifying them uniformly for presentation. A final 
problem was noted, that of anticipating questions you need to answer. 
This affects the cost-effectiveness of data capture and storage/archiv- 
ing as well as data formatting. 
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5) ?, EerriSt*:in 

Cj~.Tittee on Data Management and Computation (CODMAC) Report Review 

Tr.= viewpoints expressed by CODMAC on the whole parallel those gen- 
erctei by a variety of NASA-sponsored user- requirements studies and 
works'ops. Concerns with data quality, timeliness, and accessability 
were : re valent. This report has significant importance, in that it 
represents the views outside NASA of NASA's data distribution role. 

Two points concerning the report were noted: (2) the report reflected 

an assessrent of the space data handling situation from the perspective 
of research scientists, b) scientists are sometimes resistant to tech- 
nolccical change compared with their existing research medium largely 
because they feel unfamiliar with the new way of doing things and/or 
processing of data is beyond their control or preferred specifications. 

R. Lorrie 

Data Base Issues in Geographic Applications 

The use of the relational data base approach to handling geographic- 
ally encoded data files and graphics data was discussed. The System R 
apprcach was described. A significant point was that the system operated 
on recp^ds not sets or arrays of data. Up to date this has created 
severe overhead problems for graphical data representations, and image 
arrays have not been addressed. System R, as most DBMS, have concen- 
trated on the accounting/inventory data management problem. The rela- 
tipr.a'! data base concept would be useful for many applications if it 
cojl: e-'ficiently handle spatial oriented data other than nominally 
er.coced files. The value of the relational data base technology for 


ORltili^AL 


spitial data will depend strongly upon the ability to extract quanta- 
tive features of that data. These are necessary as a base for the 
inquiry and manipulation facilities which are key to relational systems 


7) W. Sharpley 

Problems of High Rate Computations 

The technology of high data rate computing, even for the specialized 
case of image arrays, is developing independent of NASA to serve other 
user needs. There is probably a compatability with the development of 
high rate computing where NASA may not need to ‘support hardv/are develop- 
ment for sensor data interpretation, but rather the formatting of data 
bases. A basic bottleneck in image file computing is I/O. Potential 
solutions include back end machines and floppy disks. It is important 
to note that geographical/image data bases use more streamlined DBMS, 
fonctions tnan generally provided by corr.irsrcial systems. This is because 
many of their concerns are limited in scope and do not involve require- 
ments such as frequent updating of all records in a file. 


8) E. Schwartz 

Data Base Coordinate System for the Storage and Retrieval of 

Satellite Data 

The Navy's procedure to place satellite cloud cover infonnation into 
an easily addressable working data base for meterological models to 
operate on was described. The system is a good example of the kinds of 
software and hardware architectures which can be developed to achieve a 
very specific product derived from satellite imagery. The data is not 
archived (i.e., each new overflight replaces the previous one) and the 


r- 2 p projection process is not reversible to the intial scanner data. 


Flexibility in satellite data analysis capabilities has been sacrificed 
for a comprehensive input capability to complex weather forecast models 
which require systematically formatted/mapped data to operate efficiently. 
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Houston Workshop 

The third workshop conducted by the Working Group on Information 
Utilization and Evaluation was held at the Lunar and Planetary Science 
Institute near Johnson Spacecraft Center at Clear Lake City, Texas, February 
23 and 24, 1981. The objective of this workshop was an examination of some 
of the important attributes of data which give it value as information and 
methods for evaluating the ability of remote sensor systems to provide 
accurate data/ information for resource management purposes. Attendees 
at this workshop are listed in Appendix A. Seven presentations were made 
at this workshop. TABLE WR-III is a copy of paper presentation schedule 
followed during this meeting. Summaries of these presentations along with 
important identified research issues in Section V of this report. 

TABLE WR-III 

Titles of Presentations and Speakers at 

DATA AND INFORMATION PERFORMANCE WORKSHOP 
HOUSTON, TEXAS 
FEBRUARY 1981 

PERFORMANCE PARAMETERS AND CRITERIA FOR JUDGING FEDERAL INFORMATION SYSTEMS - 
Gaylord Worden 

DATA AND INFORMATION FROM THE SRS SYSTEM - 
Raymond Jessen 

ASSESSMENT OF THE STATISTICAL ACCURACY OF ESTIMATES MADE FROM REMOTELY 
SENSED DATA - 

Patrick Odell 

MAINTAINING THE SPATIAL COMPARENT IN STATISTICAL AGGREGATION - 
Alan Strahler for Reginald Gol ledge and Larry Hubert 
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TABLE WR-III (continued) 

FEDERAL- PROVINCIAL AND INTERNATIONAL DIMENSIONS OF CANADIAN REMOTE 
SENSING PROBLEMS - 
Donald Clough 

MODELLING THE EFFECTS OF MISREGISTRATION ON MULT I -SPECTRAL CLASSIFICATION- 
Fred Billingsley 

THE DATA REPRESENTATION ACCURACY QUESTION IN REMOTE SENSING- 
Bill Cobberly 


Summary of Worden's Presentation, JSC, 

Performance ParWters and Criteria for Judging Federal 

I nf^ormati on Systems 
February ^3, 1^5T~ 

Worden began by describing the Federal Statistical System, its problems, 
and the missions of the Office of Federal Statistical Standards and Policy 
(OFSS&P). The United States has a decentralized Federal statistical system: 
many agencies produce statistics, but each is responsible for the design, 
collection, quality, and dissemination of its statistical outputs. Some 
agencies contract with private firms to collect statistics, whereas others 
have established large and professionally sophisticated organizations to 
produce statistics. 

The quality and policy relevance of Federal statistics vary widely. 
Statistics produced by small units, or as a by-product of administrative and 
regulatory activities, are sometimes unreliable, poorly designed, and 
inadequate for important policy problems. Failure to anticipate major 
data needs has led to political pressures on statistical agencies. The 
objectivity of Federal statistical data has been subject to periodic threats, 
but the record to date is good. Protection of the privacy of respondents who 
provide records, as well as minimization of the burden of response are problems 
of increasing significance today. 
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Worden indicated that the OFSS&P is the only central Federal agency con- 
ce'*ned with Federal statistics. The specific missions of the office are to: 

- Ensure the policy relevance, quality, and integrity of statistical 
data produced by Federal agencies; 

- De^'elop plans so tnat the Federal Statistical System will be able 
to .*«eet future information needs efficiently; 

- Assist in minimizing the burden on all persons and organizations 
asked to supply data to Federal agencies; 

- Xai main a proper balance betv;een protecting individual and 
business rights to privacy and confidentiality and meeting the 
information needs for public policy. 

The CF5S4? seeks to ensure policy relevance by addressing broad issues 
that cut across Department and policy decision structures. The office repre- 
sents these issues throughout the decentralized Federal statistical system, 
ard it encourages corrjnuni cations among agencies, reporting populations, and 
use »'5 so that alternative needs are recognized in agency planning. The 
objective is to promote the development of an integrated statistical system 
with sufficient uniformity in concepts and measurements that the system can 
serve current and emerging national policy needs satisfactorily. 

The 0FSS4P routinely reviews the contracts let by small units for data 
collection and other statistics-related activities. Subjective benefit-cost 
a^elys's, as v;ell as studies of objectivity, access, and use, are conducted 
by che office to maintain the credibility of Federal statistics and the 
confiCencial i cy of data on v/hich they are based. For Federal statistics tc 
be tr.. 3 ted, the public must be sure the (1) data are not manipulated to 
se*ve narro'.v political interests and (2) the objectivity of the data is 

fully protrcced. Effective protection must be given to the confidentiality 
of dac2 suoolied by persons and businesses. Regulatory agencies rarely 
pledge ccr-**dr'“bi»lity because they are concerned with the activities of 
specific er.ticies. Some actions of regulatory agencies concern the major 
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agencisS prr^jclng economic statistics because respondent cooperation declines 
with increasing fear*; of publicity. 

V.orCen emphasized that the OFSS&P seeks to address trade-offs in data 
collection, handling, and respondent burden for the entire Federal statistical 
syster.. Tne office has developed an Information Collection Budget which will b 
used to evaluate statistical reporting burdens on the public. Remote sensing 
syste~.s “ay orovide observational means that minimize respondent reporting 
burdsri, b-jt the issues of privacy, verification, accuracy, and cost remain 
impor:a-.t c-obler.s. Worden observed that the criteria used by the OFSS&P 
are appro; riate to statistical systems for natural resources, science, and 
technology, but neither concepts nor institutions are sufficiently developed 
to permit trade-off and benefit-cost analyses comparable to those under- 
taken for economic and other Federal statistics. 

Data and Information from the SRS System 
by Raymond jessen, university or Caiirornia, Los Angeles 

In Professor Jessen's talk he dealt mainly with procedures for the 
estimation of cereal crops. He began with a historical perspective. In 
the early days SRS tyoe estimates depended upon reports from farmers. 

Errors could crop up in this type of reporting so adjustments were made 
and a structure developed called the Crop Reporting Board. This board 
exa-ines information produced by state officers to make its estimates 
of production. As this board looks at prices of crops that are often 
highly speculative the integrity of their data is of primary importance 
and has created some problems in the past. This area of data integrity 
is still being v/or'<ed on today. 

In tne 1930* s some experimentation was done on the use of airphoto 
interoretation to look into cereal grain areage for: 
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0 nipping; and 

0 crop type identification. 

Each field area was visited on the ground. This data collection proce- 
dure was rot adopted but was still useful in looking at variability of 
field sizes - this was good for research in sampling. 

Next, cane research on yield. This was done in small plots in a 
n’j-ber of states. These plot data were then aggregated up to field 
size statistics. Cotton ball counts were the first really valid statis- 
tical estimates of yield. Corn was next looked at for yield. Corn was 
the first crop where a big difference between the plot data and the 
.field harvest data began to show up. (Biological yield - different 
than - harvested field yield). These research harvests developed in 
the 1930's but the procedures developed were not adopted until the 
1950's. 

We should remember that it is easy to sample in Indiana and 
Illinois but in- other areas without regular field boundaries this is 
difficult so a natural boundary scheme was worked out. But SRS is not 
just interested in yield or just in acreages. All this led to the 
current SRS systems. 

In this system farmers are visited and shown airphotos of their 
fields with boundaries overlayed. Field sizes are used as a surrogate 
for planted areas. These photos are then changed and annotated in the 
field by the faraer and the reporter. Crop and acres are the type of 
information gathered but other data are taken as well (e.g., number of 
livestock). 

This information is taken by SRS during their survey in about 
400 samples sites per state. Some states have more sites - California 
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has sorr?'' 800+, New England may have states lumped together. 

Now In June not all crops are planted (e.g.‘, corn, soybeans). 

These data ere gathered by asking what will be planted. These data then 
form the sample units for yield data. These plots are then looked at 
monthly and detailed data are forv/arded to the Crop Reporting Board; 
along with other Information, This other Information survey data sent 
to farmers v/ho are asked how is your crop compared to the norm. Later 
farmers are asked hov/ much yield does It look like. In this type of 
reporting you have early predictions which are more forecasts (crop 
not made yet); later these can be referred to as estimates. Again these 
data are reported to the state office, then to the Crop Reporting Board. 

These people use ancillary data tool Such as car loadings, (that Is, 
number of cars loaded with a specific crop and the total tonage of that 
load)., etc. Then they make essentially very judgmental decisions. 

Next, we asked Dr. Jessen why this decision system is not more systematized? 
Jessen replied that he is not sure of how subjective judgments like this 
can be made more objective. After all, if we can do mo»*e valid sampling, 
we could get a more unbiased estimate. If so, we should be able to 
get right on. What we have though is a series of: 

0 sampling problems; 

0 reporting error problems; and, 

0 r.on- response type problems (not too much of a problem 
because you can drive around fields and get data). 

Sr.i and USDA are getting some accuracy checks on specific crops. 

The ora'.ce crop Is one that has more objective limb counts (e.g., oranges- 
sq-re:s: shipped, ccttcn-gired, sugar beets-milled). These can be 

relatively accurately verified. In crops where on farm use is made 
of t‘'o rrsp such as corn for cattle feed estimate data Is less reliable. 
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Jessen was then asked - Why do we keep this system going? Why not 
change over to a new system? Jessen stated that In his opinion It was 
because they don't know the magnitude and scvrces of error and the 
problems associated with other schemes. He also stated that we must 
remember that SRS/USOA want to keep farm visits as they are collecting 
more than Just the acreage and yield data. 

Another problem area pointed out by Jessen involved the deter- 
mination of objective yields (biological yields are higher than reported 
yields). This problem Is well known within the agency (USDA). 

Although remote sensing Is currently being used In Landuse mapping 
to reduce sampling error, more work and documentation Is needed In 
this area. In the future they may use Remote Sensing to put out more 
local statistics using Landsat as a base. 

In conclusion, Jessen restated the problem that biological yield 
may be higher than reported yield and how do we get a handle on this. 

The agency Is aware of It, he says, but It Is not a straightforward 
problem to handle. He also stated that he was aware that remote sensing 
Is being used In Landuse mapping to reduce sampling errors and that In 
the future USDA/SRS may use remote sensing to put out more local statis- 
tics. For this Landsat might serve as an acceptable base. 

Assessment of Statistical Accuracy of Estimates Made from Remotely Sensed Data 
Patrick L. O'bell, University of Texas at Dallas 

By Its nature, resource Inventory from satellite remote sensing Involves 
S3"pl1ng, applications of decision theory, and most Importantly the processing 
o' enor.T.pus amounts of data. This last element In resource inventory schemes 
such as LACIE or AgRISTARS means that classical sampling techniques are 
ne arly applicable. The sensor data have a large number of potential errors 
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associated wit^ registration, atmospherics, etc., which require statistical 
rethods that are robust or entirely nev/. When considering accuracy assess- 
pent, the LACIE or AgRISTARS procedure involves three kinds of estimating 
conditions of crop yield: a) where a true value is made available to 

compare modeled estimates, b) where estimates of true value generated from 
other sources are available, c) where no grounds for comparison of modeled 
estimate exist. Condition (a) is always desirable, condition (b) can in 
part be overcome by comparing the variances of estimates generated by 
both methods, but for (c) the only practical alternative is to develop a 
modeled hypothesis and verify it on a surrogate. The verification for (c) 
can come from the use of analogue areas or use of a test set of verified 
data. 

Much of the AgRISTARS work will involve category (c) conditions, where 
nc check is available for verification. Therefore, fundamental research 
r.sads to be undertaken in looking at: understanding some radiance to 

sc-isor model parameter effects, improved statistical theory to incorporate 
spatial attributes e.g., BLOB and AMOEBA, and Improved sampling strategies 
w'lich are both more robust and sensitive to differentiating crop types. 

Tnese could not practically evolve through using sstan standard data sets 
to test models and applying systematic constraints (covariances) to the 

test one. A final need is to document the research procedures as well as 
the methodology adopted in each application to focus on commonality for concern 
of future adoption as procedures. 

What follows is my list of research topics which I feel should be 
pursued. If resolved these should lend significantly to our understanding 
and hopefully increase the accuracies of our estimates. 

(a) Develop a better technique for estimating the number of distinct 
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crops 1r. a region. We have used cluster analysis methods In the past but 
other techniques have not been applied to this problem (mode estimation 
using statistical theory of extremes). 

(b) Investigate algorithm for finding boundaries which may be 
applicable for adjusting estimates for mixed pixels, especially for small 
field application. 

(c) Perform a major study to devise various sampling plans and methods 
for different application using remotely sensed data. Sampling plans where 
probabilities of mul ticlassificatlon are Incorporated. 

I am Interested In systematic random sampling plans In which a 
systematic methodology Is Incorporated to stratify with respect to weather 
(agrophysical regions or agrometeorological regions as It were). 

Compute the bias term for all estimators. Bias remains the dominant 
problem of all production estimates. Develop what I will call attribute 
vector observation vectors (composed of O's and I's) which will allow 
better use of temporal Information. That Is, If an observation over time 
has attributes that qualify It to be that of a crop class of Interest the 
vector would contain component of only ones (zero's Imply that the pixel 
lacks an attribute of the particular crop class. Classification should 
follow using these vectors. 

Finally, I have had Interest In evaluating Information content of 
remotely sensed data to determine whether or not the data sets produced 
by remote sensing can do a specified job. ^ concern has been whether or 
not the data contains 

(a) sufficient Information to estimate acreages accurately, and 

(b) sufficient Information to separate similar crop classes such as 
wheat and barley 


The research, so far, has been primarily concerned with showing that 
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verified data whose accuracy parameters are supposedly known contain an 
order of magnitude more information for estimating proportions in a mixture 
probability density than does classified data (data processed with some 
mixture of man and machine) or unlabeled "raw" data. This research tends 
to confirm the old adage "In bridge, one peak is worth a hundred finesses!" 


Now I will get on my "soap box" 

1 . Document, Document, Document 

2. I still would like to see a formal Book of Test Data Sets 

3. I would like to see a formal book of models 

4. I would like to see a formal book of sampling plans. 

' 

In the early days of NASA, I recall several books summerizing signi- 
ficant results. Books on remote sensing has started to appear bi't more are 
needed to complete the record. 


Surenary of remarks* by Professor Donald Clough, Feb. 23 6t 24, 1981 
at Houston meeting of Working Group on Utilization and Evaluation. 


Certain elements in a data base require careful scrutiny in 
the evaluation process. A possible list would include; 

(a) integration 

(b) uniformity and standardization 

(c) time requirements of the users (e.g. real or archival) 

(c) Elusiveness of specific user needs, -- a distinction 
being made here between requir em ents , as put forward 
in ansvrer to questionnaires, and the purposes to which 
information is put. 

(e) credibility -- in the way of acceptable justification 
for the information system. In Canada, sovereignty 
imperative carried great weight. Cost/benefit analy- 
sis served only as the crutch leaned on for policy 
purposes. Language is a very important factor, and 
quantitative arguments are not nearly as potent as 
the political ones. 
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(f) Public good. Issues relating to involvement with 
private industry, matters relating to proprietary 
information and confidentiality have to be faced. 

(g) Policies and standards. While the "data bank" con- 
cept has appeal, one-step shopping for all kinds of 
intormation for all kinds of purposes seems unreal- 
istic especially in viev.’ of the lack of coordination 
among agencies as to definitions, classification, 
formats, map scale, and so on. Different agencies 
not only have their own specific areas of interest, 
but their own favored methods and standards. 

(On this point, Gaylord Worden offered the cogent observation 
chat purpose considerations cannot be emphasized enough. What, he 
asks, are the agencies trying to achieve? If there is no acceptable 
answer to this basic question, there can be no bona fide analysis, 
cost/benefit or any other.) 


Professor Clough's contribution included many observations in 
addition to his "presentation." Because the.se were extremely 
pertinent, I have tried to capture them along with the gist 
of his statement. IRH 


In the case of technology push, which characterizes remote 
sensing applications, most of the experiments in agriculture are 
not veil controlled; internal logic is not well constructed; and 
it is virtually impossible for "captive" scientific and technical 
people to be objective and uncover the linkages. 

The history of satellite technology is important and should 
not be overlooked as a source of clues to understanding how we 
got where we are and where we are heading. Canada developed mag- 
netometers. Larry Morlcy established the center as receiving 
station for Landsat I. Then came cabinet author izrt ion, criteria 
for evaluation at the political as compared with bureaucratic 
levels far different one from the other. The main desideratum 
foi Canada was " sovereignty protection ," and the Cabinet agreed 
and approved. Cost/benefit analysis came later, but it was only 
the window-dressing following the political process. In some 
kno'A-r. cases, the claims embodied in such analyses were so un- 
realistic as to prove embarassing. 


At present, Canada is interested in the Seasat concept. 
(Incidentally, satellite technology has been good for Canada's 
export narket, the building of receiving stations all over the 
world being the stock-in-trade of a Canadian Company.) The ar- 
guments in favor of SAR (Synthetic Aperture Radar) systems are 
in the political arena and not based on economics, since pro- 
ponents are well aware that 15 to 20 years will be needed for 

proper development of the system. Not only is it premature to 
try to measure benefits but at this time it is kno\'m that there 
is not market. "Users" are, in fact, scientists associated with 
the government. There being significant lack of a commercial 
market , Canada is skeptical as to whether its space program will 

ever "pay" in the conventional sense. This is not, however, to 
say that they value it less. On the contrary, the Treasury Board 
looks not only to its spinoff (such as the construction of ground 
stations worldwide) but also to certain kinds of needed technical 
capability. Canada needs SAR because of its problems with cloud 
cover; it needs a sensor able to provide illumination through 
clouds and night and day, ice surveillance in its northern re- 
gions being a natter of paramount concern. It would appear that 
at this stage, clear focus on a recognized area of concern could 
be a more promising way of demonstrating remote-sensing capability 
than diffusion through efforts to reach many users. Clough's 
advice: find a winner . 

It was Clough's considered opinion that neither NASA nor NOAA 
was equipped yet to deliver an information system and he asked 
how the message about the technology's present inadequacies 
ana shcrtccmings (cf. Billingsley's presentation) could get to 
the users. This is not a trivial question when one recalls the 
amount of "selling" and "showing-and- telling" that has accompanied 
Landsat and that we see happening in the case of the shuttle. 
Clough says that optimization cannot occur before we understand 
and address the structure of the market and for this a struc- 
tural, theoretical framework is needed, but unlikely because 
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of the bureaucratic, managerial approach which has been used. There 
can be no proper evaluation until we find some one who is willing to 
pay a price. Other>^ise. we are all operating on faith. 


MODELING MISREGISTRATION AND RELATED EFFECTS 
ON MULTISPECTRAL CLASSIFICATION* 

Fred C. Sillingsley 

Jet Propulsion Laboratory, California Institute of Technology 
Pasadena, California 91109 


Spectral analysis generally takes the form of tnultiapectral classification 
in which the clssif ication is done by comparing the sample measurement vector 
to the statistics of the set of known material vectors (training statistics) 
representing all possible classes, and by using one of several decision 
methods, determining which of the knowns it most nearly matches. 

The problem pursued will be the effects of misregistration on the accuracy 
of multispectral classification in answer to the question; 

What are the effects on multispectral classification accuracy of 
relaxing the overall scene registration accuracy from 0.3 to 0.5 
pixel? 

The misregistration is but one of a group of parameters (noise, class 
separability, spatial transient response, field size) which must all be 
considered simultaneously . The thread of the argument (which will be 
discussed in detail below) is this: any noise in the measurements (due to the 
scene, sensor, or the analog to digital process) causes a finite fraction of 
measurements to fall outside of the classification limits. For field 
boundaries, where the misregistration effects are felt, the misregistration 
causes the border in a given (set of) band(s) to be closer than expected to a 
given pixel, so that the mixed materials in the pixels causes additional 
pixels to fall outside of the class limits. Considerations of the transient 
distance involved in the difference in brightness between adjacent fields, 
when scaled to "per pixel", allows the estimation of the width of the border 
zones. The entire problem is then scaled to field sizes to allow estimation 
of the global effects. 

This approach allows the estimation of the accuracy of multispectral 
classification which might be expected for field interiors, the useful nujiber 
of quantization bits, and one set of criteria for an unbiased classifier. 
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The following conclusions were elaborated In detail In the presen- 
tation: 

• The difference between 0.3 end 0.5 pixel mieregistretion 
is in the noise for Bultiepectral clessif ication. 

• Precision users may have to re-register image segments any- 
way, making extreme registration precision by the system 

of less importance. 

• Interpolation algorithm choice is relatively unimportant, 
provided a higher-order interpolator is used. 

a If small field are important, small pixels are more important 
than sensor noise contributions. 

In addition, several observations result: 

• System registration to 1-2 pixels should satisfy users 
of film products. 

• There is a grey area of 0.5 to 1-2 pixels in which the 
requirements for high precision are not well justified. 

TH£ BASIC MODEL 

The expected effect of misclassif ication may be estimated by a simple 

first-order approach, because the differences in classification accuracy 
between the many classification schemes and conditions that have been tested 
are overshadowed by the vagaries in the data and assumptions in the 
classification process, so that higher order analysis will contribute little 
additional understanding. 

Consider first the probability of correct identification of a field 

interior pixel. Field interiors are nonuniform because of the combined 
effects of sensor noise, scaled to equivalent reflectivity (NEi5p ) and 
inherent nonuniformities in the field itself. The overall brightness 
aistribution is considered to be Caussian - this is approximately true for 
field interiors, although the distribution deviates considerably toward 
bimodal for mixed materials at field borders. 

The combined effect of these various noise sources produces a finite 
probability of misclassif ication. (Figure S-1) The first-order estimate 
considers the total variance caused by the scene, sensor and quantization as 
compared to the defined class size limits, however these are determined. 
Similar, but relatively second-order, effect may be expected with a higher 
order analysis. Proper classifier training, resulting in accurate limits, is 
essential (Hixaon et al, 1980). 

For simplicity, and because of the later desire to misregister one (or 

more) of the bands, the discussion will assume that spectral bands as sensed 

will be used, and that for recognition, the unknown pixel must fall between 
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«p-ropri*te liaits in «vtry band tested. Therefore, brightness outside of e 
licit in any one band is sufficient for rejection, so that ve need to consider 
only one band at a tine. 

The probability of a saople being within the class limits can be derived 
by assueing that an ensemble of clean signals from a series of areas of the 
same material can be anywhere within the quantizing range with* uniform 
probability, but Chat individual samples are perturbed by the Gaussian noise 
with a distribution equal to C • The probability distribution of the sign.-'.! 
plus noise is found by convolving the probability distributi >n of the signal 
with that of the noise. The probability of correct class assignment (i.e., 
the pixel is within the class limits) is then found by integrating the 
probability distribution between appropriate class limits (Friedman 1965). 
The result of this calculation is shown in Figure S'^2. In the useful range 
of y9(3<^<7), the curve can be approximated by 

/Jlog P - - 0.40 

where P - probability of correct classification, and 

, with class size and a in the same units. 

scene 


class size 


scene 


Sources of noise will be the scene itself and the sensor, both assumed to 
be random for this analysis. The root mean square (rms) sum is taken to give 
the total effective noise. A number of pixel measurements ’ may be averaged 
together to reduce the noise before classification. This final noise figure 
may be compared to the width of the class to give , from which the 
probability P of correct classification may be estimated. This leads to the 
Classification Error Estimator, Fig. S-3. 

As an example, consider a scene having a field-interior variation of 32, 
to be viewed with a sensor having a total noise figure of 12. The total 
effective noise seen by the classifier (upper left) will be the rms sum of 
these, or 3.162, which for a total 0-255 digital number (dn) range, would be 
8.1 dn. If the class width (determined by the classifier algorithm) is 25 dn 
(right center) the /3 ■ 3.1, giving P ■ 0.742 (right lower). If this P is 
not accurate enough for the analysis, s :veral pixels must be averaged(right 
upper): a 2x2 averaging will raise p to 6.2, giving a new P ■ 0.86. 

Considering in this way allows an estimation of the total noise 
permissable as it affects the attainable classification accuracy. If the 
au.ount of scene noise to be encountered in a given classification task can be 
estimated, the allowable extra noise from the sensor and quantization can be 
specified by estimating the loss of accuracy of the classification caused by 
quantization error. This leads to an estimate of the number of bits which 
will be useful. 


Define the perfect sensor as having no random noise nor quantization error 
(i.e., an infinite number of bits). This will define (for nxn pixels averaged) 
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for the reel eemoTi b®cauee of the finite ^icneor ^quentization* 

The new probability of correct claaaif ication P ia related to by: 

P • P 

o 

A plot of the loat in classification accuracy vs* Pq is given in Figure 
S-4, for the paraaeter faailies and sensor^ ^pc.ene* Noise 

allocation starts with defining the desired P^ and ascertaining that the 
required obtained. Definition of the allowed ^P determines (e.g., 

from the graph) the allowed ^sensor^ ^ scene* ^ estimation of the scene 
noise for which the other conditions apply allows the calculation of the total 
sensor noise allowed. The final step is to partition this noise between 
sensor random noise and quantization noise. 

For example, let the desired Pq ■ 65Z and allow no more than 2Z loss 
■dua to the total sensor noise. The no-sensor-noise must be S5.7 to give 

Pg. Then, from Figure S-4, the allowed ^sensor " ®*® * ^ scene* 
the scene has a ^ scene “ 2Z, the allowable ^sensor “ 0.6x22 ■ 

1.22, which must be partitioned between NE A p and the qua ntization noise. 
For NE Ap ■ 12, the allowable quant " - 1^ ■ 0.662, which 

can be met by 6-bit quantization. 

Two observations are important here: (1) Increasing the number of bits of 

quantization produces improvements which asymptotically approach zero, as each 
successive bit reduces the step size by a factor of 1/2. (2) A scene having 

as little as 22 variation is a very uniform scene. Since this noise is rms'd 
with the sensor noise, it will overwhelm any but a very noisy sensor. 
Therefore, for purposes of multi-spectral classification, more than six bits 
would seem to be unnecessary. 


EDGE EFFECTS 

To this point, the analysis is based on pixels well inside uniform fields 
and well away from field boundaries. A number of experimenters have spent 
appreciable time discovering that classification accuracy falls off at 
boundaries due to what has become known as the mixed-pixel effect. We will 
start at that point and attempt to model the effect to allow us to quantify 
our expectations. 

We assume as a starting point that all the spectral bands used in 
classification, whether obtained from one date or series of dates, are in 
perfect registration. This means that when the pixel grids from each band are 
aligned the data concents (field borders, roads, all features) are also 
aligned - note that this is more than simply having all internal distortions 
removed, which is all Chat most geometric rectifications accomplish. 
Misregistration will (later) be considered as the lack of alignment of the 
pixel grids; because the computer can only work with pixel grids, aligning 
these pixel grids because appear to the computer as a shift in the 
boundaries. We will assume that training samples are accurate and that class 
limits have been set from these by the classifier chosen. The classification 
is modelled as follows: signature shifting in any individual band will tend 
to cause misc lassif icat ion, so that the situation may be treated one band at a 
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,vioe. The effects of pixel sixture in all bands aay than ba ras'd together if 
desired. The entire analysis simplifies to the consideration of the transient 
intensity shift across field boundaries as conpsrad to the class limits and 
the noise components of the measurement. 

The first step in analyzing the spatial extant of pixel mixing across 
borders is to estimate the shape and extent of the transient intensity shift. 
Zf the impulse response functions or the modulation transfer functions (HTFs) 
of the various components (and, hence, the entire system) are known, a precise 
transient response may be calculated. For example, the specifications for the 
Thematic Mapper for I ;i!sat D call for a 2X to 98X time equivalent of about 2 
pixels implying a IvZ-90Z transient response of about 1.3 pixel. The 
practical result of this is that the "infinitely sharp" edges of the real 
scene will be softened by the tiltcring effect of the scanning aperture 
(assumed to be rectangular and having uniform response) and it is this 
softened transient response which is sampled. Interpolation required for 
registration will cause some further softening, and the use of any of the 
competent higher-otder interpolation functions (sinx/x, TRW cubic convolution, 
modified cubic convolution, other splines) will have minor effects of the rise 
time. A total '^10-90 (transient response from lOX to 90X) of 1.5 pixels 
with no ringing will be used as a surrogate global value. 

The transient situation across a border is sketched in Fig. S-5. We are 
concerned here with the decrease in probability that a given pixel will have a 
value within the class limits as that pixel moves toward the boundary, as 
shown in Figure S-6. The analysis only needs to determine the area under the 
normal curve (assuming the noise is Gaussian) between the limits as determined 
by the classification class size and the offset from the "field interior value" 
caused by the mixture. The important scaling involved is the amount of signal 
shift caused by the transient total shift T, as related to the desired class 
size S, for a given /3 . The left portion of Figure S-7 reflects this shift in 
brightness (vertical axis) as it affects the area within the class (the 
probability of recognition). 

The transient rise distance estimated for the Thematic Mapper has very 
close to a Gaussian shape and a ■ 1.5 pixel. The amount of 
brightness shift is the difference between the brightness of the field under 
consideration and the adjacent field which is causing the shift. The 
important intensity relation is the magnitude of this shift, T, as related to 
the size S of the class being tested by the ratio T/S. These curves, for 
various T/S, are combined with the probability curves of the previous 
discussion in Figure S**7. From this may be estimated the loss in probability- 
in classification of pixels near borders. 


BIAS IN FIELD SIZE ESTIMATION 

It can be appreciated that several things are happening simultaneously: 
If the lower limit of field B and the upper limit of field A have a gap 
between, pixels "lost" by field B will not be picked up by field A, and will 
be considered unknowns and not be counted in either field. The lost pixels 
will be some interior pixels, due to insufficient ^ , and a large number of 
near-border pixels, resulting in apparent field size loss. Only if the lower 
limit of field B and the upper limit of field A are coincident will 
pixels lost from one field be picked up by the other, and vice versa, to give 
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cOB^lstc accounc of all pixels. For the field aixe eatimator to be unbiased, 
che lose *and-pickup in both directions Bust cancel; that ia, on the average 
Che true border euiC be locacedt The total effect will depend on the ratio of 
the number of border pixels to the number of field-interior pixels, and hence 
is a function of the field shape and size. 

This leads directly to the required algorithm for field aixe astination: 
First divide the scene into blobs, each of which ia sufficiently unifom, and 
with closed boundaries. Then for each blob (field) deteraine the average 
brightness for all the interior pixels which are safely away froD the border. 
For each segment of the border, the correct field edge decision level is 
midway (in o *s) between the average brightness of the two fields on either 
side. After the borders are located using this criterion, Che field interiors 
may be reclassified using the classification limits as determined from che 
training samples. 


EfFZCTS 0? MISREGISTRATION 

In preparation for estimation of the misregistration effects, an analysis 
will first be made of the expectations of registered data and the sensitivity 
to che various parameters estimated. The starting model used has rectangular 
fields aligned with the pixel grid. Pixels are grouped into four zones: 1) 

Interior (i)-those with centers 2 or more pixels inside borders, 2) Inner 
border (ib)-pixels with centers 1-1/2 pixel inside borders, 3) Outer border 
(cb)-pixels with centers 1/2 pixel inside borders, 4) Exterior border 
(xb)-pixels outside the borders, with centers 1/2 pixel outside. Estimates of 
classification accuracy for each zone are obtained from Figure S-7. The total 
estimate of classification accuracy is the sum of pixels in each zone 
multiplied by the corresponding zone accuracy estimate. Later, the field will 
be misregistered , changes in the number of pixels in each zone calculated, and 
che probabilities again summed. The following parameters are required: 

r - the field shape ratio, length of long, side/length of short side 
T - transient brightness difference between field being considered 
and its neighbor 
S - decision class size 

T - transient distance for lOZ to 90Z response 
P - class size S/a of Gaussian noise 

The following global values selected for the parameters are considered to be 
representative: 

r . - 2 

T/S - I to 5 

T ■ 1.5 pixels 

ft • 3 to 5 

After the parameters r, T/S, T , and A are selected, the resultant (from 
Fig. S-7) probabilities are substituted for the brightnesses in the various 
zones to produce a "probability image" aligned with the desired output pixel 
grid. The probability assigned to a pixel at a given location represents the 
probability that that pixel will have a brightness falling within Che 
classification limit determined by the classifier, for the given spectral 
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^nd. The cotal probtbility of correct clasiificecion is given by 

P • — ^ I + PjwHju P V + P w" w 1 

rnj I i '^Ib ib ^ob ob *^xb xb J 

where is the field width (short si^e) in pixels, and n£, n^i^, ng^, 

n^^ are the nucber of pixels in the various sones* Using these values, the 
global estimate of the probability of correct clasaif ication with no 
■isregistrscion is given Figure S^’S for three values of T/S. The predominant 
effect is Che pixel mixture (the effect of T/S). As expected, this is worst 
for sosll fields (n; small) because of the larger percentage of border 
pixels for these fields. Note that for T/S ■ 1, decision level midway between 
brightnesses of adjacent fields, no probability loss occurs, even with small 
fields. Unfortunately, this desirable condition cannot be systematically 
obtained . 


MISR£CIST?.AII0N OF CONGRUENT FIELDS 


The initial model for misregistration is a displacement of d pixels, equal 
in both X and y. The result of this misregistration is that some area is lost 
from Che external border, causing a further classification accuracy decrease. 
The misregistration loss as seen by the external border loss is given by 


L? 


Pxb 


d 


r 4 1 1 

r nj 


♦ 



The basic character of this misregistration loss term is l/n^, so that it 
will have a slope approximately equal to -I on a log-log plot vs nj. The 
precise results depend critically on the values of p^^ estimated for the 
Pjj^ from Figure S-7: 



Using these values. Che loss AP due to displacement misregistration is 
plotted in Figure S-9 for various parameter combinations. 


MISREGISTRATION DUE TO NON -CONGRUENCE 
1.) SIZE AND RATIO (ASPECT) CHANGES 

Size and aspect ratio changes can come about from several causes such as 
scan velocity or altitude changes, and if uncompensated can cause additional 
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stracion trrora* Progrtasiv* Biartgiacracion from a point of accurate 
^giftrecion will be caused by both causes (Figure S*lOa); the modeling of 
chis effect considers first that sice changes M ■ n'/n will cauae a ahift in 
points n to points n* both vertically and horizontally, and then that changes 
in aspect ratio will cause further shifts in the horisontal position of 
vertical borders by changing the field shape ratios by the factor R ■ r'/r. 
The resulting shifts arc: 


An^ ■ (N - 1) ny and ■ (NR - 1) my 


For analysis, this shift will be divided around the borders synmictrically as 
optimum field registration is accomplished (Figure S-lOb). Two cases must be 
distinguished (using scan velocity as a surrogate cause): 

Case 1: A slow scan decreases pixel spacing and puts more pixels into a 
given field. When these are placed into the output grid, the field appears 
stretched. The field as defined by the other (correct) bands now covers only 
part of the stretched field, so that the classification tends to see only 
interior pixels, and the accuracy, will increase, ultimately reaching the 
field-interior accuracy. The sizes of the border errors are: 

■ 2 ^1 •nd ^2 * 2 ^^1 


Case II: A fast scan has the opposite effect, causing the field to appear 

scalier and the analysis pixels defined by the other bands now include more 
exterior pixels. The classification accuracy will decrease. 

For fast scan, the smaller apparent field covers an area expressed as a 
fraction of the total: 

^2 

f t. • I. ^ . (5 1 ^ . ■ (Int.rlor) 

1 rnj 

- 2Nni -f 2KRn^r 4 - 4 (External Border) 

^xb " rSJ 

The total expccte' probability is 
^tot " ^iPi. ^ ^xb Pxb • 

Since the external border pixels are now included within the analyzed 
field, but with a low probability, the fractional area RN^ represents 
approximately the fraction of the basic field-interior accuracy to be 
expected. Since the total size shrinkage (in pixels) is small for small n^, 
only larger n^ need be considered, and the l/n| term may be dropped. 
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I Thil allows ba approxinatad for r ■ 2 by: 

j' ^*tot « ♦ -~Pxb 

( For large ficlda, the probability ia actn to bt independent of field aite, and 
only weakly dependent (becauae of low Pxb^ email aiaea* 

2.) WAVY BORDERS AND MULTIPLE ACQUISTIONS 

' For aingle-band analyaie, with bordera diatorted ao that there are pixela 

both inside and outside of the analyzed area, aome pixels will have increased 
( probabilities of correct classification and some will have less. The decrease 

I in probability across border is (very) approximately linear, so that the 

(signed) average displacement will model the effect. 

For nulci-band analysis, those pixels having a low probability of 
classification will have the largest effect as the net probability at each 
pixel location is the product of the probabilities obtained for each 
acquisition (band). In this ci se the nas displacement will produce a better 
model of the effects. 


SOME OBSERVATIONS. 


I. ON BASIC CLASSIFICATION 

e The total noise figure (compared to the class size in a given 

determination) controls fl , and in turn controls the maximuin 
attainable classification accuracy. However, for practical range of 
3 < (I < 7, increasing p has only a moderate effect. 

e Because of this, if small fields are most important, the reflected 

energy might be more profitably be divided into smaller pixels, even 
at the expense of NEAp • As this will cause an increase in data 
rate, optimum coding should be investigated. The possible nois^ 
introduced in reconstructinK the data will cause some further 
decrease in the overall effective NEAp and ao decreases p. But 
since there is smaller sensitivity to p than to l/n^. there should 
be a net gain in utility. 

a Increasing the number of bits of quantization produces improvements 
which asymptotically approach zero, as each successive bit reduces 
the step size by a factor of 1/2. 

a A scene having as little as 2Z variation ia a very uniform scene. 
Since this noise is rms'd with the sensor noise, it will overwhelm 
any but a very noisy sensor. Therefore, for purposes of 
multi'spectral classif ication, an extreme number of bits would seem 
to be unnecessary. 

II. ON EDGE EFFECTS , 

a For accurate field size estimation, the decision brightness must be 
halfway between the brighnesses of the fields on either side of a 
given boundary. This means that classifiers set for material 
identification will in general produce errors in field size. But the 
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f icld-inttrior brighcnt** is increasingly hard to tatiaata for aaall 
fields because of the fewer interior pixels. 

e It is iaportanc to kesp the transient response distance and the 
acconpanying saaple spacing smalls to get as aany pixels into a given 
ground distance as possible. Field area errors becoae large at n^ 
■ 5 or less. The transient distance must also be matched between 
spectral bands. 

e At the resolution expected for the Thematic Mapper, the atmospheric 
point spread function may become more dominant than the Thematic 

Mapper point spread function. If this is determined to be true, the 

registration requirements may be relaxed since scene-dependent 
registration will be required anyway. 

III. ON KISRZGISTRATION 

e For large T/S (i.e., 2 or more) the edge effects arc so great that 
the base probability is drastically affected, and the external border 
pixels have zero probability of being within the class limits. For 
this reason, there is no misregistration effect for large T/S. 

o Square fields show the most misregistration loss, when scaled to 

'■•I* . . . 

e A shape ratio r"2 is believed to be representative. 

e Misregistration loss decreases with higher /} . However, these losses 

in general are small to begin with, and the discussion calling for 
srcrificc of /} to gain smaller XFOV (more pixels n^ into a giver, 

field) would seem to override. 

e Increase in t decreases the basic accuracy of edge pixels and also 

increases the misregistration losses. 

0 Ccocetric rectification and registration procedures must not only 

remove the internal distortions but must also produce pixels on a 

defined (preferably ground-referenced) grid. Current procedures do 
not do this. Without this reference grid, users will have to 

re-intcrpolate before multi-temporal data can be compared. 

e Scale and aspect ratio errors will have only minor effects on 
moderate-area problems. But they will cause problems in correlating 
over large distances. 

e Altitude relief displacement will require users to use many control 
points to register images in areas of high relief. 

e Unless standard reference grids are established, users requiring 

registration will have to interpolate every image, even in low relief 

areas. 

e For single-band analysis, the algebraic average of the displacement 

may be used. For multi-band analysis, with erratic errors in 

location among the bands, the lowest probability of correct 
classification holds and the rms of the displacements is appropriate. 


CUiC‘ 

OF POOR QUALITY 


-C45 


AN UNANSWERID QUESTION 

This rtporC aodtls the potential nisregistration effects on ■ultiapcctral 
classif ication accuracy. It nay allow the cor.^srison Oi the various tests and 
siBulations, and points out the variables which Bust be reported for those 
siBv«lations to allow their validation. It does not answer the folloviiiji 
question: Given a certain loss in accuracy due to Bisregistration, how does 

that daaaga the ability to use the data analysis results? These evaluations 
will be discipline dependent, and Bust be sought separately. 

REFERENCE 
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Summary of a Heuristic Method for the Comparison of Related Structures 
by 6olledge and Hubert as presented by Strahler. 

Department of Geography, University of California, Santa Barbara, CA 


Basically this work involves a strategy to evaluate data analysis 
schemes that are supposed to clarify the structure underlying a set of 
proximity measures. 

As Alan sees it, there are four categories of possible applications: 

(1; Given two different analyses based on a single data source, does 
either represent the data significantly better than the other? 

(2' Given 2 analyses from a single data source, v/hich is closer to a 
specific theoretical structure assumed to underlie the data? 

(3) Given 2 theoretical structures and one analysis does either represent 
the analysis better than the other? 

{4} Given a single analysis based on one data source, is the information 
present in the data accounted for satisfactorily by the analysis? 

The original basis for this type of work was laid out in a method developed 

by ’.;olfe (1975, 1977) 


?j,v-w 


P uv - Puw 
2(1-Pvw) 


Pvw i 1 


u,v,w = 3 random variables with some joint distribution, in which 


Var(V) = Var(w) 

Ih..5 , Puv = ruw if and only if Pu,v-w = 0. 

This rear.s that a test of equality betv.een Puv & Puw reduces to a test of 


original pass is 
OF POOR QUALITY 

- This test can be perfonned under normality. 
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- But, we can also use RANDOMIZATION to test this nonparametrical ly. 

- To do this,. we estimate Pu,v-w as ru,v-w 



- we form a vector of values V-W, and permute randomly, forming many 

r ' s 
^u,v-w ^ 

- Then v/e corpare ^ viith the distribution obtained by permuting 
U, and attach a significance level based on these permutations. 

- If significant, conclude that r and r are different. 

uv uw 

Detai Is : 

- There are N! permutations of U. (N is number of samples). Instead of 
enurerating r. „ for all permutations, just do a reference sample. 

- V and W rust have equal variance. Therefore, standardize them. 

Extension cf Matrix Concept 

- Assu-.e set S of n objects, and there proximity matrices A, and C 
defined cn S x S. Assume main diagc ’1 = 0. 

- Define r, ~ =* simple correlations between elements of A or elements of B. 

r ,0 — — 

- Replace elements of B and C by their Z-sums. 

- To estimate distributions of r^ permute more __ columns simultaneously 


in A. 
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REMOTE SENSING EXAMPLES 

(1) Divergence: (A.H.S.) 

- Let V be a classification of training data based on K channels; let W 
be a classification based on K+1 channels. 

- Assign V-W = class no. when classes agree; assign V-W when classes 
do not agree. 

- Use 0 coefficient instead of r between U and W-W; permute U to establish 
reference distribution. 

- Problem of degrees of freedom--difference may be significant due to 
degrees of freedom— ability to fit a higher-order model. However, if 
difference significant, this is very helpful. 

(2) Classification (A.H.S.): 

- How many classes are there in these data? 

- Let K classes number from 10 to 50. find "best" K using randomization 

'‘A.Di-I.i 

- "0" is distance matrix between each point and class of which it is 
supposed to be a member 


> 
i ■ 

! ' 


.1 
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- Compare U-W with U-r 

UtU-W. 

- Permute U to get distribution of r 

u,u-w. 

- Assign confidence level to r^ based in sampled distribution. 

- (Note that this could be used to compare alternative models as well). 

The Sections which follow, summarize the discussions of important 
research issue growing out of these workshops and present our working 
groups conclusions concerning area of research need. This is followed by 
a section listing specific prioritized research areas demanding attention. 
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V.'ILM Un: LEAK-NED ABOUT YOUP. rNFOMMAi [ON MEMOS 


Jo«ef Cihlar* 

Canndn Centre for Remoti' Si-Msin>; 
Ottawa, Ontario. 


i. IMTRODUCTION 

Tne procurement of accurate end representative intelligence about infor~- 
nti.on needs of remote sensing data users is a difficult task. A major reason 
for this difficulty is the high diversity t>f the user community and its i;ifoimac inn 
rev[uireinents , both betx»een disciplines and among users/user agencies within 
disciplines. Secondly, the needs are net constant hut evolve with time in 
cuant-tative as well as qualitative terns. Thirdly, the in format ion requirt- 
r.ents are often difficult to quantify in the manner apprepriate for rer.v>tc 
sensing technigties. 


ff 


I 

t? 

I 

I? 


Discussions with managers and specialists involved in resource r.anageme:.t 
are a very effective means of obtaining reliable Information concerning in- 
formation needs. It is for this reason that the Planning Cominirrec of this 


conference decided to give the participants an opportunity for feedback .about 
their views and concerns. A questionnaire (see Appendix) was prepared prior 
tj the conference to ensure unifori.iity in discussion:'* of varLou:. groups. T'c..- 
questionnaire was distributed fol lov.’ing the banquet on ?9 October to 16 tn.alws 
and each group was asked to atswer the questions based on their knowledge and 
experience. The discussion lasted abnnr one hour. A precedent I'nf this forrrt 
has been set at two recent annual meetings of the Sc L ! Con.servat ion Seei.-ty 

At.er ica , 


■>.-.bar of rhe Plarnit..^ Conmitr-e of t’no Coni'' rence 
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2. RESULTS 

2 . 1 Participants* Rackgrouads a n d Expecie nce 

According to their professional orientation, the 132 discuss loii 


participants could be divided into nine groups; 

Agriculture, Rangeland, Fanning 12 % 

Forestry 9 /i 

Soils ^ 

Hydrology, Water Quality, Civil Engineering 5 Z 

Ecology, Wildlife 9 % 

Geography, Land Use, Urban Planning 17 % 

Geology, Geomorphology 10 

Photograininetry , Education, Remote Sensing 18 % 


Other (Business, Economics, Meteorology , Oceanography) 6 7. 


The participants in«iicated the foliowing dof'.rees of t xporier; 
ill using remote sensing products: 


Satellite Products; no exporienre 13 " 

small cxperoince 19 " 

large- cxperien»:e -8 7 

Airborne Products: no experience 't 

small experience 2S 

large exporfince f>8 


It was also evident that the types ot remote sciisiug products uj;uii 
digital satellite data being used Much nore frcdio-nt L. tii in digital 
dat.i (note that one person could have v.oiked i. i t h I'.'t’u tvues): 


d ii 'i-rc J , 
airh rv 


Satellite ilatn used; digital fiinii 57 t 

photographic form 69 7, 

Airborno data used: digital foi-m ^6 7 

photographic form 91 7 


From the above statistics, it appears that this conference attracted 
largely professionals v;ith some degree of remote sensing e;<pertise. Individual 
resource disciplines were well represented. Since t’ne typo of v;ork performed 
by the participants was not addressed in the questionnaire, it is not possible 
to determine the relative proportions of management personnel and technical 
specialists . 

2 . 2 Resource Infonnation Requirements 

Question I concerned information which are not provided by existing 
methods but could be supplied through satellite aiu! airborne rt.:.ote sensing. 

As expected, a wide variety of information needs were listed. The lists were 
subsequently reviewed to summarize common rcqulrerient s under separate heniiincs. 
.\lthough an attempt v;as made to obtain clear and specific descriptions of in- 
formation needs, the answers xvere vague in numerous cases, flic results are re- 
produced below in a form consistent with the original responses, A slash (/'• 
is used to indicate chat the requirement was stated by mi>re than one group ru: 
not dll groups used the modifier following the slash. 

ORIGINAL PAGE fS 
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2.2.1 Agriculture , Rangeland. Farmin ;; : 

real or near-real cimo information tprcferahly .Tt pre- 
visunl tines) on v;eeds. insects, defol Lai io:i , disease, 
noiscuro stress, iu-rbiclde ami ferrilL*'et m is-app 1 icat ion ; 


(i) 
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(il) £iccur;ilc , C ime ly/nmm.-i 1 crop (.ill im.i i 

by producclon iireas rolevanc For ptivntf nii'l public 
decisions ; 

(iii) information required for planting decisions 
(planting rates, fertilizer applicat ioiid , field 
conditions, nearby weed infestations); 

(iv) range forace production and gra5;ing capacity, in- 
ventory, trend assessment over large areas am! 
location of brush in pastures; 

I 

(v) crop yield prediction through modelling for bot!» 
domestic and international crops; 

(vi) pipeline construction effect on long- tori!: crop 
prodvivC t ion; 

(vii) qiiai'.ti tative information on residue amounts; 

(viii) more accvirate i/enther p red ic.r ior. . 

2.2.2 To res try : 

(i) infestatiou and plant stress; 

(ii) staLewide/amiu.al timber volume mon i i ur iug inelu- 
dini; fi'i'est depletion; 

(iii) rate of tropical de fores cat ion ; 

(iv) forest energy ns.sessi.u- 't ; 

(v) fuel type condition motii i or Lng; 

(vi) betf.T subdivision of forest Inm! .over c t..p rio-. 

2.2.3 Soils and Geology ; 

(i) erosion: critical arena, statewide ero^.ion das-? 

changes, ef feet ivonetis of cnnsi-rvat i on ii;ea -ui res ; 
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(il) (lc[iCh Co bedrock niul to water Cable; 

(ill) surflclal and shallow auhsurface (lralna*;e; 

(Iv) soil moisture over large areas; 

(v) rapid soil mapping in poorly accessible areas, and 
mapping vegetation and rocks in support of soil mapping; 

(vi) location and mapping of subsurface minerals. 

2.2.4 Hydrology, Water Quality, Civil Engineering * 

(i) surface water parameters: depth, suspended solids, 

pH, alkalinity, heavA' metals, P 2 O 5 content, and lake 
surface temperatures over large areas; 

Cii) snowpack water equi •alent, and snow depth; 

(iii) irrigated acreage b crop, type of i rr i r.al Lou 
system, and source f water; 

(ivl regional hydrology vonitoring (Wyoming, Montana) ; 

(v) accurate flood plo u delineation, and areal coverage for 
flood events; 

(vi) ice hazard detection (oceans) ; 

(vii) land water resourcis data. 

2.2.5 Ecolo gy . md Wildl ltc : 

(i) biomasj; estimation; 

(ii) vegetation napping over large, renoce .'.r,. >•; 111 ‘ 

mapping, »)ver snail ai..is at large se.iK-; 

(iii) habitat cover m.acs for large areas; uoivlfiring 
habi-.i'' cb.anpes; 

(iv) mappie.g age cla es in. ’..rush e‘.'r.mim i r f i-w ; 


h 
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(v) 

(vi) 

(vii) 


wctlanJii plnnr cypos invi-ntnry; 
chnngiis in critical coastal zone 
success of nines rec lar..at ton . 


areas ; 


2.2.6 Geography, Land Use, Urban Platinino ,: 

(i) statewide land cover naps; 

(ii) better land mapping methods; 

(iii) level II land use maps of urban areas and change de- 
tection in urban areas; 

(iv) land cover (at 5 acre resolution) of crop types and 
crop rotation Cor soil erosion assessment. 


2 . 3 Technology Requirement s 

Numerous items listed in response to Oue.stion J at • more appropri- 
ately characterir.ed as technology requirements. Tin; first fiv'v wi*i»- men- 
tioned by more than one group: 

(i) all weather data for timely app 1 icat iotis ; 

(ii) improved spatial resolution; 

(iii) increased coverage frequency; 

(iv) better timeliness, near-real or real time iiu-luc;in" 
fast data delivery; 

(v) continuity of data; 

(vl) satellite data reception in the I'irid; 

(vii) worldwide, consistent coverage; 

(viii) reliable data foniints and h»^tt-.-r distribut «•>*■ 
remotely sensed data (a library of imag^ i desire- 
able), and better infnrmation nr iirvtg** •'•iiliiy h.-fi.te 


purchase ; 
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(tx) intefir.Jt ion of cxiailn}' ;:^rounil nod nirnrufr dnln; 

(x) whole-frame re;^lstratiun, a".d lietl.-?- r< rat ion of 
ocean scenes; 

(xi) ocher spectral bands (niid-IR, thermal IR) ; 

(xii) interface Landsac data with geographic information 
systems, and improve transferability of data; 

(xiii) package remote sensing techniques; 

(xiv) make available user-oriented products; 

(xv) nationwide data base for resource applications; 

(xvi) inexpensive, common format mini-processing systems; 

(xvii) regional environmental raoniroring centres. 

? • ^ Maj or Obstacles 

OuesCion 2 requested a list of "major <d>starlc:; which must he ueer- 
come in order to make remote sensing a practical and useful t**ol iu your worl:". 
The first 10 items listed below were mentioned by more chan one discussion 
group: 

(i) continuity of data; 

(ii) lower costs; 

(iil) connuinicnt ion gap between rcsenrcliers «nnd u.sers, jti- 
cluding helLer puhliiity !\.r npp L Lc.it i nu;-! of digiiil 
data; 

(iv) cheap, subaidir.ed data tliat. ch.e user c.m afford; 

(’■' recognition that remote sensing is a suppler eiu to 
existing systems and rarely -i sui»st i t i:t e ; 

('■!) long-term, active federal r .vc;ilc;r.ent t ) support an 
.operational system, and better teehno lo-v trin*'<r 
procedure'? : 
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.(vii) 


(vUi) 


(ix) 

(x) 


(xi) 


(xii) 


(xtii) 


(xiv) 

(xv) 


(:;vi) 


(xvi i) 


(xvi ii) 


umlursCanJ Iiig and accept anci* uf f lu- ni-ed f\»r rcmclc scnsiii;; 
methods by people in key positions and the provision of 
support ; 

Institutional problems: disagreements coiicemiu)' leading 

role and lack of coordination naionp. ajtonoies; 

locally available expertise in renote sensing; 

recognition that remote sensing offers products which need 

promotion (currently non-existant) to be funded; 

end user must know how to use rcmot»;ly sensed data ns well as 

their advantages and 1 1ml -.at ions ; 

better education at universities and (?) liigh schovils, 
and general public awareness; 

opportu:iities for training and for iisors to work wiLij 
remotely sensed data on their own prob1<‘T.s and In tlieir 
own cnvironnvn ts ; 
lack of funds and time; 

technology not ilirectrd at tlic grass routs U-vvl and 

too much vnpliasis on high toclmo! ; 

unwillingness of tiie private sccto*- to commir to new 

technology, atid nnwi I 1 1 ngi>w-«s of tin-> gov*.- rmrunr c • i>s- 

plorc technology transfer to InJusi rv; 

tiser cluut in setting priority for romur<> sensing dat.i 

availabil ity; 

technology transfer at ni l-.nr. cost and Ic* (•••?.( . 


tltnclv data for all u.scrs. 


~ - 
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An .nccuratc aiuJ conprchenslve spufl f iiMrion i<r In l\>i r .iLlcm roiiu I rcr.cnt 
suc<‘ei»s fully be provliled in a brief disinisslon siirli ns rcpf-rtuil here. .Never-* 
3 s, Che lisc of information rcqutrev.ents ^Ivon nhove has two unique 
.'is;ir‘'.cs. First, tlie roquirenenCs ropresent problems of « -.irrcnt concern c.> 
reso-.rce managers. Secondly, the list vas compiled by a broad spectrum of 
resource managers and is therefore relevant for a ranpe of disciplines and 
resources . 

T'ne major obstacles and many technological requirep’.ents were identified 
quite clearly and consistently. It seems evidonl that although remote sensing 
methods have shown considerable potential for rc.sourcc rinnagenent applications, 
a lot more work must bo done in mnkiup. the appropr i nti- r.et’novis available to 
agencies an*l individt.als responsible for operational Vv’ork. Onlv when this 
accomplished will renore sensing yield the anticipated, ho’.i'f i l a f r«’ra Impro*. 1 
resource management. 

Many participai'.ts were* also concerned ahoiit the cost .'f future remot<jly 
sensed data. The requirement for inexpensive data was echoed bv the mijorlty 
of discussion groups. This is an important eons Lderar ion , especially nov; •.•her. 
plans for operational satellite remote .sensing are In-lnj* finalized in the 
United States. 
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DRAFT LETTER FROM IDA HOOS REC'D 2/27/81 

Understandably, scientists and engineers charged with the responsibility 
of devising and designing satellite and other technology capable of responding 
to future needs would like a scientific basis for evaluating present data 
systems and assessing the value of the Information they can deliver. At first 
glance, this might appear to be a relatively simple research task; one would 
have only go to the users, ascertain their needs, and learn about their 
present and prospective requirements. 

I 

(a) The above may look reasonable but Is entirely unrealistic. It 1s based 
on nonexistent premises and untenable assumptions, not least among which is 
the one that might best be called technical optimism . Most of the "capability" 
of Landsat, for example, to serve as a unique source of Information suoerlor 

to any other Is still In the q.e.d. category; It Is still to be demonstrated. 
The "test cases" do not yet stand up well under test. The most ardent advo- 
cates are not users willing to pay for systems and service but "brokers" and 
technical middle people who have a vital role in the process of technology 
transfer but who are pushers rather than users . 

(b) Approaching the natter from the viewpoint of political reality , one might 
go so far as to state that we cannot honestly evaluate the systems In place 
for fear of perturbing precarious relationships or offending particloating 
agencies. When, for example, NASA has worked out a difficult cooperative 
arrangement with the Veteran's Administration for the promotion of some 
device, "research and evaluation" are regarded with trepidation for fear some- 
thing other than a valentine be produced. The "user needs surveys" conducted 
In conjunction with the NASA-NOAA transfer, the presentations at our workshops, 
and our cumulative experience as professionals and academics should convince 

us thiat in order to perform a proper analysis we must establish social ground 
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truth as painstakingly and meticulously as do the technical people in their 
assessments. We also know that neither time nor circumstance allows for 
such a course of action. 

(c) In a "management environment" where cost/effectiveness is the iron 

law, justification for present and especially future generations of satellite 
technology by reference to value and returns is tantamount to quantifying 
pie- in- the- sky. It was difficult enough to compile creditable numbers to 
satisfy Senator Proxmire. Nor was it an easy task with the traditionally axe 
wielding 0MB. What we can expect from not only Mr. Stockman but even from 
the public- at- large is already the handwriting on the wall. In an era where 
NASA's stock is low, it is unrealistic to count on big investments and 

quixotic to make claims that cannot be substantiated. Technology 

cannot be rationalized by non-existent demand. The public has its 

own perceptions of punities and in the present political climate not all 
premises carry the same amount of credibility. 

(d) Information on the international scene appears to carry more threat than 
promises. Books like The Geopolitics of Information (Geoffrey Smith, Oxford 
University Press, 1980), reports like "Information and the New World Order," 
and sessions of world organizations attest to the enormous concerns being 
generated by the gathering and dissemination of information about resources. 
Nations are concerned about the possibility of "economic espionage." New 
alliances and interdependencies are foreseen. Simply to offer all countries 
equal acce^^s at the same cost as is to act in a discriminating manner. These 
are factors that must be taken into account in the calculation of cost/benefit 
ratios of future generation of satellite technology. 

But this brings us back into the morass of factors impeding proper access 
lent of the use and value of information. And this probably explains the 
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dearth of such evaluations. The social context renders them mutually impos- 
sible. But, is this to say then that none should be done? Quite the contrary. 
Starting with the premise that the social contest i_s crucial, we might do 
well to consider another social context — not the one that serves as a kind 
of gridlock but one that provides us with what seems to be a generally 
agreed upon paradigm for the future. The world new of the future as envisioned 
in the Qlobal 2000 report might be a good framework. It is the global situation 
needs of the year 2000 and on to which tomorrow's technology must respond, and 
it is clear that remote sensing and the information it can provide would be 
a key factor in meeting the challenges which transcend all national borders and 
internicine warefare among government agencies. Because it is the far 
horizon that has more certainty and for which there is greater agreement than 
the present, we might consider it as the framework and thus unfetter ourselves 
from preoccupation with fabricating a story designed to please rather than 
to enlighten. 




Energy, Mines and 
Resources Canada 


Energie. Mines et 
Ress^rces Canada 
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Science and Technology Science el Technologie 
CANADA CENTRE FOR REMOTE SENSING 


December 9th, 1980 


four Ml Vom nlMunci 


Out m Nom xtMfmci 

5072 


Dr. J. Estes, 

University of California, 
Santa Barbara, California, 
93106, U.S.A. 


Dear Jack, 

Attached is a manuscript "What He Learned About Your Information 
Needs" which was written on the basis of the group discussions at the 
Kansas City Conference. Given your current interest in information 
requirements and processing, you may find this useful. 

Best regards. 


Josef Cihlar, Head, 

J^lfcations Development Section. 


Attachment (1) 
JC/ma 


2464 Sheffieid Road 
Ottawa Car^ada 
K1A0Y7 


2464. rue Sheffield 
Ottawa Canada 
K1A0Y7 
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United Slates Deoarlmcrit 


of the Interior 


{>V.O\ (i!' \l S' K- : 1 


In Reply Refer To: April 7, 1981 

ZG3-Mail Stop 710 


isorandun for the Record 

Meeting report 

Professor John E. Estes, Dep?.rtn.ent of Geography, University 
of California, Santa Barbara, California 

April 3, 1931, 3:00 to 4:00 p.m. 

To discuss topics for basic research in accuracy of digital 
classification. 

Jack Estes introduced our conversation with the information that he is involved 
with XASA on the process of information, utilization, and evaluation of digital 
ciassitlcatioo of Landsat data. He is looking for research projects that will 
carry into the 1990' s. 

I first discussed that there is no standard for defining accuracy and that it 
c>£ans different things to different people. Hard and Brooner, 1976, PE&RS, first 
dc;uraeated an approach to accuracy as the lower of the two-siJad confidence 
Halts about the observed sample proportion correct. I consider accuracy to be 
the observed sample proportion correct, with associated confidence limits. 

Brian Berry recommended to Fitzpatrlck-Lins, 1980, PE&RS, that the lower one- 
siJe ! confidence limit be established about the observed sample proportion 
correct when the errors of ocmlssion are not considered. Berry (1979), in his 
review of ay draft paper, did not necessarily agree with the point-in-polygon 
accuracy test, and stated that his local computer systems should be used. 

I then recommended, as a long range research project, that he propose bringing 
dijltal classification to the status of a mature mensurational system. I stated 
m/ concept that a mature mensurational system is one where the mathematics and 
statistics are completely rigorous and general in application; where the error 
budget is completely known and understood; systematic errors are calibratlonal 
and correctable; random errors are known and their effect can be propagated; and 
finally, that the system Is completely documented as to theory, analysis, and 
computer program. I referred him to o>’ paper, "The Role of Software in Photo- 
graametry," 1967, PE&RS, for the philosophy involved for a mature mensurntlonal 
system. That paper concercs positional statistics, but the concept is applicable 
to thematic remote sensing. I also referred him to my paper, "The Ballistic 
Camera Accuracy Review Project,*' 1964, PE&RS, for consideration of the massive 
effort involved in upgrading a mensurational system, and bringing it to the 
mature stage. 


Subject: 

Visitor: 

Tine of visit: 
Purpose of visit: 


OF POOa QUALITY 
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I Chun discussed other smaller research problems: 1) investigating jthe probabil- 
ity distribution for systematic sampling since the binooilal distribution is 
based on simple random saitpllr^; and also to consider sampling in tvro frames, 
a;> vs do in the Ling sampling algorithm. 2) considering the entire classifica- 
tion error matrix, Instead of only the diagonal elements as most people do. 

R>V Mead and staff at VPI&SU use one approach to studying the entire matrix, 
and I an looking at another approach. Both a single matrix from a given accuracy 
Leit should be studied, as well as several matrices from an experiment to study 
cifferent factors. A rigorous approach is needea to interpret the entire content 
oi the classification error catrL< on a probability basis. 

Estes requested a copy of the paper "Summary Tables for Selected Digital Image 
Processing Systems," V. Carter and others, 1977, USGS Open-File Report 77-41^, 

He showed me in a document he had prepared for NASA (he will send me c copy), 
aa illustration of a concept for analyzing a digital classif iratlon systcia. He 
stated that that section of the document discussed what we had talked abrut, 
but did nut put it as succinctly as the concept of a nature mensurational 
system. 


I wished him luck on his project* 


George H. Rosenficld 


cc : Witmer 

Guptlll 
Place 
Wray 

Rosenf ield 

Estes 

RCR 
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Intarnational SutlnaM MaehInM Corporation 



S55 Ballay Avanua 

P.O. Box S0C20 

San Joaa. California 951 S3 

June 2, 1981 



Pr. John Estes 
Geography Department 

University of California, Santa Barbara 
Santa Barbara, California 93106 

Reference : Your Memo Dated May 18, 1981 


Dear Jack, 

If Nevln Bryant agrees, I believe that the material which I 
sent him on the referenced letter should be incorporated in 
the summary notes on the meeting at Santa Teresa Lab. A par- 
ticular point to be made is that NASA should not spend its 
precious funds on research in data base management systems 
since there are so many other forces which will move these. 

On the other hand, I feel very strongly that demonstration 
systems need to be funded, both for the purpose of making data 
available, and secondarily, to provide a test bed for resolving 
practical (non-baslc research) issues in data base management. 

The following applies to the preliminary siunmary of proposed 
research areas: 

1. Information and decision processes: 

I believe the most likely approach 
and therefore the first priority is 
that associated with expert systems. 

In order for the general public or 
laymen ’’government agencies" to 
obtain value from satellite data, 
a great deal of highly technical 
assistance must be given. I believe 
the expert systems approach would be 
ideal and would make a very useful 
demonstration project. At the same 
time, this area is considered to be 
one of the m.ore advanced areas of 
computer science discipline. 
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I would be very doubtful that work to 
build an economic theory which treats 
information as a commodity will yield 
anything useful. Information is an 
entity which allows the better management 
of other more tangible and valuable assets . 
The value of Information thus depends 
very much on what assets one is able 
to manipulate. I believe NASA would be 
very unlikely to obtain payoff from such 
work. 

2. Data and Information Attributes: 

I am concerned that all of the areas of 
research here are very dependent upon 
the particular context and application. 

I wcuM strongly recommend that these 
areas be pursued within the scope of 
a demonstration project such that 
concrete examples can be found. I am 
not hopeful that much of general value 
can be established. 

3. Data Base Management and Use: 

■'*ere, as said in my letter, I believe 
f^iat NASA money should not be spent 
on basic computer science work since 
this field has so much activity in 
that area. Rather, the money should be 
foucsed on demonstration projects in 
which the research element would be 
investigations into the particular data 
type that comes from satellites. I 
believe all of the elements listed in 
this category would be good topics within 
a particular demonstration project. 

4. Data and Information Systems Performance: 

All of the areas listed here look useful and 
I have no sense of priority about them. 

As you can tell from my multiple suggestions 
of it, I believe the key to progress in this 
area is well-chosen demonstration projects 


'V.. 
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Dr. John Fstes 


-3- June 2, 1981 


which attempt to advance the state 
of practice, if not research, in 
reducing and providing data to end 
users. Since the future behavior of 
end users is so uncertain, I believe 
it is important to have demonstration 
projects to "do market research". I 
believe it is very unlike' y that academic 
research can really isolate where the 
values are in such a broad and complex 
future context . 


Sincerely, 


Glenn Bacon 


GCB : cap 
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SUMMARY BREAKDOWN OF INDIVIDUAL 
PARTICIPANTS BY EMPLOYMENT 


PRIVATE INDUSTRY 14 

FEDERAL GOVERNMENT 16 

STATE AND LOCAL GOVERNMENT 4 

UNIVERSITY 22 

TOTAL INDIVIDUALS PARTICIPATING 56 
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INFORMATION AND DECISION PROCESSES WORKSHOP 
ASILOMAR, CALIFORNIA 
SEPTEMBER 1980 


ATTENDEES 

GLEN BACON, IBM Corporation, Santa Terresa 

ROBINSON BARKER, St. Regis Paper Company, Jacksonville, Florida 

JAMES T. BONNEN, Michigan State University 

NEVIN BRYANT, NASA Jet Propulsion Laboratory 

CHRISTOPHER CLAYTON, University of California, Santa Barbara 

LUDWIG EISGRUBER, Oregon State University 

JOHN ESTES, University of California, Santa Barbara 

OSWALD GANLEY, Harvard University 

ROBERT MACDONALD, NASA Johnson Space Center 

MICHAEL J. MCCORMICK, Planning and Community Affairs, State cf Washington 

C. BART MCGUIRE, University of California, Berkeley 

ROBERT POTTER, Water Resources, Sacramento, California 

LEONARD SLOSKY, Science and Technology, Colorado 

CHARLES VARS, Oregon State University 

DIANA WATTS, Space Science Laboratory, University of California, Berkeley 
DARRELL WILLIAMS, NASA Goddard Space Flight Center 


REMOTE SENSING FOR RESOURCE MANAGEMENT 
SPECIAL SESSION MEETING 
KANSAS CITY, KANSAS 
OCTOBER 1980 


ATTENDEES 


WILLIAM ANDERSON, Technicolor Graphics, Sioux Falls, South Dakota 

MARION BAUMGARDNER, Laboratory for Applications of Remote Sensing, Purdue University 

NEVIN BRYANT, NASA Jet Propulsion Laboratory 

JOSEF CIHLAR, Canada Center for Remote Sensing, Ottawa, Ontario 

JOHN ESTES, University of California, Santa Barbara 

PEGGY HARWOOD, National Governors Conference, Washington, D.C. 

WILLIAM MACFARLAND, University of Missouri, Columbia 
EARL MERRITT, Earth Satellite Corporation, Washington, D.C. 

GENE THOMPSON, Missouri Farm Association, Jefferson City 
CHARLES VARS, Oregon State University 
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PLANNING SESSION #1 
SANTA BARBARA, CALIFORNIA 
APRIL 1980 


ATTENDEES 

HUGH CALKINS, State University of New York, Buffalo 

CHRISTOPHER CLAYTON, University of California, Santa Barbara 

GEOFFREY DUTTON, Harvard University, Computer Graphics Laboratory 

LUDWIG EISGRUBER, Oregon State University 

JOHN ESTES, University of Califonria, Santa Barbara 

ROBERT MACDONALD, NASA Johnson Space Center 

DENNISON PARKER, United States Department of the Interior 

RONALD SHELTON, Michigan State University 

DIANA WATTS, Space Science Laboratory, University of California, Berkeley 
CHARLES VARS, Oregon State University 


PLANNING SESSION #2 
CLEAR LAKE, TEXAS 
JULY 1980 


ATTENDEES 


GLEN BACON, ISM Corporation, Santa Terresa 

NEVIN BRYANT, NASA Jet Propulsion Laboratory 

CHRISTOPHER CLAYTON, University of California, Santa Barbara 

LUDWIG EISGRUBER, Oregon State University 

JOHN ESTES, University of California, Santa Barbara 

FORREST HALL, NASA Johnson Space Center 

IDA HOOS, Space Sciences Laboratory, University of California, Berkeley 
ROBERT MACDONALD, NASA Johnson Space Center 
BRUCE SCHEER, The Planning Economic Group, Boston, Massachusetts 
CHARLES VARS, Oregon State University 


DATA BASE USE AND MANAGEMENT WORKSHOP 
SAN JOSE, CALIFORNIA 
JANUARY 1981 


ATTENDEES 

DELMAR ANDERSON, Central Intelligence Agency 
GLEN BACON, IBM Corporation, Santa Terresa 
RALPH BERNSTEIN, IBM Palo Alto Scientific Center 
NEVIN BRYANT, NASA Jet Propulsion Laboratory 
JOHN ESTES, University of California, Santa Barbara 
STAN HANSEN, Boeing, Seattle, Washington 
ADRIAN HOOKE, NASA Jet Propulsion Laboratory 

IDA HOOS, Space Science Laboratory, University of California, Berkeley 

RAY LORRIE, IBM Research, San Jose 

ROBERT MACDONALD, NASA Johnson Space Center 

BOB MYERS, IBM, Yorktown Research Center 

GENE RICE, NASA Johnson Space Center 

EVE SCHWARTZ, Fleet Numerical U.S. Navy, Monterey, California 
WILLIAM SHARPLEY, ESL/IRW, Mountain View, California 
DAVID SINNOTT, NASA Research Center 
DON WALKETT, Turra-Mar, Palo Alto, California 


PLANNING SESSION #3 
HOUSTON, TEXAS 
FEBRUARY 1981 


ATTENDEES 


NEVIN BRYANT, NASA Jet Propulsion Laboratory 

LUDWIG EISGRUBER, Oregon State University 

JOHN ESTES, University of California, Santa Barbara 

IDA HOOS, Space Science Laboratory, University of California, Berkeley 

ROBERT MACDONALD, NASA Johnson Space Center 

CHARLES VARS, Oregon State University 
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DATA AND INFORMATION PERFORMANCE WORKSHOP 
HOUSTON. TEXAS 
FEBRUARY 1981 


ATTENDEES 


FREDERIC BILLINGSLEY, NASA Jet Propulsion Laboratory 
NF.VIN BRYANT, NASA Jet Propulsion Laboratory 
DONALD J. CLOUGH, University of Waterloo, Ontario, Canada 
WILLIAM COBBERLY, University of Tulsa, Tulsa, Oklahoina 
JOHN ESTES, University of California, Santa Barbara 
CECIL HALLUM, NASA Johnson Space Center 

IDA hGOS, Space Science Laboratory, University of California, Berkeley 

GLENN HOUSTON, NASA Johnson Space Center 

RAYMOND JESSEN, University of California, Los Angeles 

PATRICK O'DELL, University of Texas, Dallas, Texas 

ALAN H. STRAHLER, University of California, Santa Barbara 

CHARLES VARS, Oregon State University 

GAYLORD WORDEN, Department of Commerce, Washington, D.C. 


INDIVIDUAL CONTACTS OUTSIDE OF MEETINGS AND WORKSHOPS 


CARL BRONTHAVER, Soarks Commodities 

REGINALD GOLLEDGE, University of California, Santa Barbara 

ALAN MURPHY, Oregon State University 

ALAN OETTINGER, Harvard University 

GEORGE ROSENFELD, Department of the Interior/USGS 

TERRANCE SMITH, UCSB/Carnegie Melon 

JOHN ZUMBRUNN, Conmidities Corporation 
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5TEERING COMMITTEE 


GLEN BACON, IBM Corporation 

NEVIN BRYANT, NASA Jet Propulsion Laboratory 

CHRISTOPHER CLAYTON, University of California 

LUDWIG EISGRUBER, Oregon State University 

JOHN ESTES, University of California 

FORREST HALL, NASA Johnson Space Center 

IDA HOOS, Space Science Laboratory, University of California 
ROBERT MACDONALD, NASA Johnson Space Center 
BRUCE SCHEER, The Planning Economic Croup, Boston 
RONALD SHELTON, Michigan State University 
CHARLES VARS, Oregon State University 
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BREAKDOWN OF INDIVIDUAL PARTICIPANTS BY EMPLOYMENT 


PRIVATE INDUSTRY 14 

FEDERAL GOVERNMENT 16 

STATE AND LOCAL GOVERNMENT 4 
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TOTAL INDIVIDUALS PARTICIPATING 56 



AREAS OF EMPHASIS 


• AS SHOWN ON HEADQUARTERS VIFWCPAPH 
Data Base Structures 

Identification of Data/Information of Significant 
Economic/Social Value 

Alternative Methods of Using Remote Sensing 
and Performance Evaluation 


• AS DEFINED BY INFORMATION UTILIZATION AND EVALUAIION 
STEERING COMMITTEE 

Information and Decision Processes 

Data and Information Attributes 

Data Base Management and Use 


Data and Information Systems Performance 



AREAS OF EMPHASIS (con,t) 


• INFORMATION AND DECISION PROCESSES 

Current and Probable Future Institutional Arrangements 
Informational Components of Decision Systems 
Decision Processes 

• DATA AND |NFOR^tATION ATTRIBUTES 

Develop Methods for Assessing Data Attributes 
hueraction of Data Attributes 


Effects on Future Decision Processes 


AREAS OF EMPHASIS(con't) 
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• DATA BASE MANAGEMENT AND USE 

Input of Remote Sensor and Other Data 
Storage, Retrieval and Management Issues 
Interaction and Transmission of Stored Data 
Types of Products Demanded of Future Systems 

• DATA AND INFORMATION SYSTEMS PERFORMANCE 

Performance Parameter Definition 
Performance Parameter Assessment 


Estimation of Performance Parameters in Constrained Situations 
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TWO MODES OF INFORMATION USE 
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MAJOR RECOMMENDATIONS FROM KANSAS CITY DISCUSSIONS 


• EXISTING INFORMATION SYSTEMS: THE USE, USERS, AND RELATED 
PROCESSES NEED TO BE REVIEWED AND DOCUMENTED BEFORE 
FUTURE SYSTEMS CAN BE EVALUATED 

- Review and Documentation Should Include Systems with 
Other Than Obvious Remote Sensing Applications 

— Review Process Should Place Special Emphasis on Why, 
How, and What the Effects of Multiple Data Sources are 
on Operation of Existing Information Systems 
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PRELIMINARY SUiVMARY OF PROPOSED RESEARCH AREAS 


• INFORMATION AND DECISION PROCESSES 

- Development of a General Theory of the tconomics of 
Resource Management Information Systems 

- Develop Models at Varying Levels of Aggregation of 
Information Systems with Public, Private' and 
International Components 

- Investigate Analytic Systems and Processing for Information 
Extraction from Data Bases on Disparate Concepts/Data 
Explore Potential Linkages Between Artificial Intelligence 

- Oriented 'Expert Systems’, Image Analysis Logic and Data 
Bases for Resource Management Decision Making 

- Augument Current Research in Economic Theory which 
Treats Information as a Commidity 
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PRELIMINARY SUMMARY OF PROPOSED RESEARCH AREAS 
(con't) 


• DATA AND INFORMATION ATTRIBUTES 

- Develop Improved Techniques for Measuring Tradeoffs 
Between Use of Existing and New Information Systems 
for Renewable Resource Decision Making 

- Investigate Impacts of Timeliness, Reliability, Accuracy and 
Assured Product Delivery on Cost Benefit Potential 

- Examine Alternative Means of Assessing the Public Good 
and Multiple Use Aspects of Data/Information 

- Explore Use of Existing Collateral Data to Reduce 
Requirements for Sensor and Ground Truth Data Acquisition 


PRELIMINARY SUMMARY OF PROPOSED RESEARCH AREAS 
(con't) 
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• DATA BASE MANAGEMENT AND USE 

- Factors Influencing Data Input and Interaction Potential of 
Remotely Sensed Data. Examine Impacts of: 

data base structure on input and archiving formats 

improve query capabilities processing rates on 
systems development and use 

data availability, archiving and the opportunity costs 
of data storage 

decentralized data bases/information systems on 
cost/benefits ratios 

- Improvement Of Processing Stratagies For Integration Of 
Disparate Data Types, (ey Issues Include: 

understanding generic functions in spatially oriented 
data manipulation 

effects of positional accuracy on estimates derived 
from multiple data planes 

investigate performance and capacity requirements 
for the large record size and special purpose processing 
required for imagery and geographic applications 
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PREIIMINARY SUMMARY OF PROPOSED RESEARCH AREAS 
(con'l) 


» DATA BASE MANAGEMENT AND USE (con't) 

— Examination of Possible and Probable Future Environments 
which will Impact Renewable Resource Data Base 
Management and Use 
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PRELIMINARY SUMMARY OF PROPOSED RESEARCH AREAS 
(con't) 

1 

I 

k 

• DATA AND INFORMATION SYSTEMS PERFORMANCE 

— Examine Fundamental Aspects of the Accuracy of 
Products of Remote Sensing 

- Develop Relevant Measures which Characterize 
Data/Information (e.g.. Minimally Sufficient Statistic) 

- Develop Procedures Necessary for Renewable Resources 
Satellite Sensor Systems to Achieve 'Mensuration Systems' 
Status for Given Applications 

• 

— Continued Examination of Fundamental Aspects of Sampling 
Theory as They Apply to Spatial Distributed/ Temporally 
Varying/ Renewable Resources Parameters; at Local, 
Regional, National, and International Scales. Key Issues 
Include: 

• greater sensitivity to deviations from 'ncrmal conditions' 

• identification of key parameters and determination of 
correlation coefficients between remote sensor scene 
derived data and ground conditions for use when no 
independent source of verification exists 

• explore the potential of nonparametric test of data 



PLANNING SESSION #1 
SANTA BARBARA, CALIFORNIA 
APRIL 1980 
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HUGH CALKINS, State University of New York, Buffalo 

CHRISTOPHER CLAYTON, University of California 

GEOFFREY DUTTON, Harvard University 

Computer Graphics Laboratory 

LUDWIG EISGRUBER, Oregon State University 

jOHN ESTES, University of California 

ROBERT MACDONALD, NASA Johnson Space Center 

DENNISON PARKER, United States Department of the Interior 

RONALD SHELTON, Michigan State University 

DIANA WATTS, Space Science Laboratory, 

University of California 

CHARLES VARS, Oregon State L)niversit>' 
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PLANNING SESSION #2 
CLEAR LAKE, TEXAS 
lULY 1980 


-F18- 


f 

I 

I 

I 


GLEN BACON, IBM Corporation 

NEVIN BRYANT, NASA jet Propulsion Laboratory 

CHRISrOPMER CLAYTON, University of California 

LUDWIG EISCRUBER, Oregon State University 

JOHN ESTES, University of California 

FORREST FIALL, NASA lohnscn Space Center 

IDA HOOS, Space Sciences Laboratory, University of California 

ROBERT MACDONALD, NASA Johnson Space Center 

BRUCE SCHEER, The Planning Economic Group, Boston 

CHARLES VARS, Oregon State University 
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INFORMATION AND DECISION PROCESSES WORKSHOP 
ASILOMAR, CALIFORNIA 
SEPTEMBER 1980 


GLEN BACON, IBM Corporation 

ROBINSON BARKER, St. Regis Paper Company 

JAMES T. BONNEN, Michigan State University 

NEVIN BRYANT, NASA jet Propulsion Laboratory 

CHRISTOPHER CLAYTON, University of California 

LUDWIG EISCRUBER, Oregon State University 

JOHN ESTES, University of California 

OSWALD GANLEY, Harvard University 

ROBERT MACDONALD, NASA Johnson Space Center 

MICHAEL J. MCCORMICK, Planning and Community Affairs, 

State of Washington 

C. BART MCGUIRE, University of California 

ROBERT POTTER, Water Resources, Sacramento, California 

LEONARD SLOSKY, Science and Technology, Colorado 

CHARLES VARS, Oregon State University 

DIANA WATTS, Space Science Laboratory , 

University of California 

DARRELL WILLIAMS, NASA Goddard Space Flight Center 


INFORMATION AND DECISION PROCESSES WORKSHOP 
ASILOMAR, CALIFORNIA 
SEPTEMBER 1980 


THEORY OF INFORMATION SYSTEMS: PUBLIC AND PRIVATE 

DIMENSIONS - 

C. Bart McGuire 

IMPLICATIONS OF GLOBAL REMOTE SENSING SYSTEMS; 

A DISCUSSION - 

Ludwig Eisgruber, leader 

INFORMATION SYSTEMS USED IN PUBLIC AND 

PRIVATE DECISION MAKING - 

Bob Barker, Bob Potter, Leonard Sloski 

ARE THERE COMMON RESEARCH NEEDS?; ADISCUSSION - 
Charles Vars, leader 

CRHICAL ANALYSIS OF EXISTING INFORMATION SYSTEMS - 
James Bonnen 

CULTURAL, LEGAL AND POLITICAL IMPLICATIONS OF GLOBAL 

INFORMATION SYSTEMS - 
Oswald Ganley 

CRITICAL ISSUES RESEARCH: A DISCUSSION - 
Ludwig Eisgruber, leader 
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REMOTE SENSING FOR RESOURCE MANAGEMENT 
I SPECIAL SESSION MEETING 
KANSAS CITY, KANSAS 
OCTOBER 1980 


WILLIAM ANDERSON, Technicolor Graphics 

MARION BAUMGARDNER, Laboratory for Applications 

of Remote Sensing 

NEVIN BRYANT, NASA Jet Propulsion Laboratory 
JOSEF CIHLAR, Canada Center for Remote Sensing 
JOHN ESTES, University of California 
PEGGY HARWOOD, National Governors Conference 
WILLIAM MACFARLAND, University of Missouri 
EARL MERRITT, Earth Satellite Corporation 
GENE THOMPSON, Missouri Farm Association 
CHARLES VARS, Oregon State University 
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DATA BASE USE AND MANAGEMENT WORKSHOP 
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SPACE APPLICATIONS DATA SYSTEMS PROGRAM 
Nevin Bryant for Peter Bracken 

PLANETARY SCIENCE DATA ACQUISITION - 
.Adrian H.ooke 

REQUIREMENTS AND CHARCTERISTICS OF SCIENTIFIC DATA BASES - 
Stan Hansen 

CARTOGRAPHIC DATA BASES - 
Delmar Anderson 

COMMITTEE ON DATA MANAGEMENT AND COMPUTATION 

REPORT REVIEW - 
Ralph Bernstein 

DATA BASE ISSUES IN GEOGRAPHIC APPLICATIONS - 
ay Lorrie 

PROBLEMS OF HIGH RATE COMPUTATION - 
William Sharpley 

DATA BASE CONSIDERATIONS FOR NAVY WEATHER FORECASTING - 
Tve Schwartz 
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PERFORMANCE PARAMETERS AND CRITERIA FOR JUDGING FEDERAL 

INFORMATION SYSTEMS - 
Gaylord Worden 

DATA AND INFORMATION FROM THE SRS SYSTEM - 
Raymond jessen 

ASSESSMENT OF THE STATISTICAL ACCURACY OF ESTIMATES MADE 

FROM REMOTELY SENSED DATA - 
Patrick Odell 

MAINTAINING THE SPATIAL COMPARENT IN STATISTICAL 
AGGREGATION - 

Alan Strahler for Reginald Colledge a.nd Terrence Smith 

FEDERAL-PROVINCIAL AND INTERNATIONAL DIMENSIONS OF 

CANADIAN REMOTE SENSING PROBLEMS - 
Donald Clough 

MODELLING THE EFFECTS OF MISREGISTRATION ON MULTI-SPECTRAL 

CLASSIFICATION - 
Fred Billingsley 

THE DATA REPRESENTATION ACCURACY QUESTION IN REMOTE 

SENSING - 

Bill Cobberly 
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INDIVIDUAL CONTACTS OUTSIDE OF MEETINGS AND WORKSHOPS 


CARL BRONTHAVER, Sparks Commodities 

REGINALD COLLEDCE, Univ. of Calif., Santa Barbara 

ALAN MURPHY, Oregon State University 

ALAN OETTINCER, Harvard University 

GEORGE ROSENFELD, Department of the Interior / USGS 

TERRANCE SMITH, UCSB /.Carnegie Melon 

JOHN ZUMBRUNN, Commidities Corporation 
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ASILOMAR WORKSHOP CONCLUSIONS CONCERNING RESEARCH 
AREAS 


• NO GENERAL THEORY OR METHODOLOGICAL FRAMEWORK EXISTS 
FOR ESTIMATING THE VALUE OF INFORMATION SYSTEMS 

e FACTORS AFFECTING GENERAL THEORY DEVELOPMENT INCLUDE: 

— Lack of a Real Market Value for the Output of Most 
Public Informations Systems 

- Quality is Based on Multiple Characteristics 

— Some Information is a Public Good, and Private Value 
may be Different from Social Value 
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ASILOMAR WORKSHOP CONCLUSIONS CONCERNING RESEARCH 
AREAS (con't) 

• LIMITED HIGH QUALITY RESEARCH HAS BEEN CONDUCTED 
IN EACH AREA 


• ONLY RUDIMENTARY MODELS HAVE BEEN DEVELOPED, TESTED, 

AND EMPIRICALLY ESTIMATED 

' I 

• SEVERAL APPROACHES HAVE BEEN EMPLOYED BUT CONTROVERSY 
CONTINUES CONCERNING OPTIMUM METHODOLOGIES 

• NO METHODOLOGY HAS PROVEN SUCCESSFUL IN GENERAL 
APPLICATION TO PUBLIC RESOURCE DATA SYSTEMS 

9 DIFFERENT APPLICATIONS CONCERNING APPROPRIATE 

METHODOLOGIES RAISE MORE QUESTIONS THAN THEY ANSWER 

• MANY SOURCES OF DATA ARE AVAILABLE TO DECISION MAKERS 
FEW REGARDED AS CERTAIN, THEREFORE CHOICES AMONG DATA 
TYPES MUST OFTEN BE MADE WITHIN UNCERTAIN ENVIRONMENTS 
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ASILOMAR WORKSHOP CONCLUSIONS CONCERNING RESEARCH 
AREAS (con't) 


• DEVELOP A GENERAL THEORY OF THE ECONOMICS OF 
RESOURCE MANAGEMENT INFORMATION SYSTEMS AND THEIR 
MODIFICATION 

• DEVELOP RELEVANT MEASURES OF DATA AND INFORMATION 
AND EMPIRICAL METHODS FOR INVESTIGATING INFORMATION 
SYSTEM PERFORMANCE 

• DEVELOP INFORMATION SYSTEM MODELS COMPRISED OF PUBLIC, 
PRIVATE AND INTERNATIONAL COMPONENTS 

• INVESTIGATE ANALYTIC SYSTEMS AND PROCESSING STRATEGIES 
FOR EXTRACTION OF DISPARATE INFORMATION FROM DATA BASES 

«» ANALYZE DATA AND INFORMATION DISSEMINATION SYSTEMS 
INCLUDING STUDIES OF 

- Accessibility — Confidentiality and Property Rights 

— Decentralization - Public Good 

— Economies of Scale - Joint Use of Data 
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SAN JOSE WORKSHOP 

KEv POINTS FROM STAN HANSEN'S PRESENTATION 
^'Requirements and Characteristics of Scientific Data Bases' 

.» WE ARE NOT JUST COMPUTERIZING EXISTING DATA AND 
PROCEDURES: WE ARE ACTUALLY CREATING NEW DATA TYPES 

• MOST COMPUTER SYSTEMS ARE VERTICALLY INTEGRATED 

TO PERFORM SPECIALIZED TASKS. WE NEED A GLOBAL TRANSFER 
CAPABILITY TO FACILITATE TRANSFER THROUGH HORIZONTAL 
INTEGRATION 

• RESEARCH IS NEEDED ON: 

Decentralized Data Base Structures 
Distributed Data Processing 
Common Carriers 

IF USERS ARE TO HAVE FLEXIBLE RESEARCH AND DESIGN 
ENVIRONMENTS 

• USERS DO NOT HAVE REQUIREMENTS 

THEY HAVE REACTIONS TO SITUATIONS IMPOSED UPON THEM 


( 
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SAN JOSE WORKSHOP SUMMARY AND CONCLUSIONS 


• DATA BASE MANAGEMENT AND USE IS DEVELOPING 
COMMESURATE WITH OVER \LL EXPANSION OF CAPABILITIES IN 
THE COMPUTER AND COMMUNICATIONS FIELDS 

• THIS DOES NOT MEAN THI FIELD IS ACTIVELY SOLVING 
PROBLEMS CONFRONTED BY REMOTE SENSING TECHNOLOGY 
IN THE RENEWABLE I'.'SOURCES DISCIPLINE AREA 

• CASE STUDIES ARE A PRACTICAL MEANS FOR DETERMINING 
ESSENTIAL COMPONENTS OF DATA BASE USE AND 
MANAGEMENT FOR EARTH RESOURCE APPLICATIONS 

• NASA MUST MAKE RENEWABLE RESOURCES NEEDS UNDERSTOOD 
TO THE COMPUTER SCIENCE COMMUNITY 
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SAN JOSE WORKSHOP SUMMARY AND CONCLUSIONS 
1 (con't) 


• COMMERCIAL MARKET HAS MOVED RAPIDLY IN THE AREA OF 
DATA BASE AND QUERY AND WILL CONTINUE TO EXPAND THE 
SPEED AND EFFICIENCY OF CATALOGUE AND QUERY CAPABILITIES 

• NASA DATA ADMINISTRATION NEEDS SHOULD BE MADE KNOWN 
TO STANDARDS BOARDS AND COMMITTEES 
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SAN JOSE WORKSHOP PRELIMINARY RESEARCH AREAS 


• IMPROVED METHODS FOR LOCATING AND ACCESSING LARGE 
VOLUMES AND LARGE RECORDS OF DATA WITHIN EXISTING 

• DATA BASES 

• EFFICIENT METHODS FOR COMBINING DIVERSE DATA SETS 

• SYSTEMS LEVEL COMMUNICATION (e.g., TRANSPORTABLE 
LANCU AGES/UNIVERSAL TRANSLATORS) 

• INTEGRATION OF HIGH DATA RATE SYSTEMS AND DATA BASES 

• MANAGEMENT OF GEOREFERENCED DATA BASES 

• STANDARDIZATION OF DATA. IN PARTICULAR DEVELOPMENT OF 
COMPREHENSIVE HEIRARCHY OF LAYERED STANDS PERMITTING 
GLOBAL INTERCOMMUNICATION OF PRODUCTS WITHIN AN 
OPEN SYSTEM 


SAN JOSE WORKSHOP PRELIMINARY RESEARCH AREAS (con't) 


o ARCHIVING AND PURGING OF DATA BASES 

• DATA AGGREGATION 

• DATA VALIDATION 

• INTERACTIVE MAN-MACHINE ENVIRONMENTS AND THE ASTHETICS 
OF DISPLAY 

• EXAMINE USER REQUIREMENTS AND CONSTRAINTS WITHIN THE 
CONTEXT OF CURRENT SYSTEMS ENVIRONMENTS TO PROJECT 
FUTURE NEEDS AND USES 
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SAN JOSE WORKSHOP PRIMARY RESEARCH AREAS 
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• PERFORMANCE AND CAPACITY IMPROVEMENT 

- Ability to Handle Longer Records and Special Purpose 
Processing Associated with Imagery and Geographic 
Applications 

- Ba:>ic Structures of Data Base Management Systems 

• DATA ADMINISTRATION 

— Managment Strategies and Data Ownership 

• data and QUERY 

— Improved Cataloging Capabilities 

— Improved Query Capabilities (e.g., Employing Hueristic Logic) 

• SYSTEMS STUDIES 

- Development of Models for Base Generation and Use 

— Problems with Data Handling and Trend Analyses (e.g., Logic) 


PRELIMINARY SUMMARY AND CONCLUSIONS DRAWN PROM 
I HOUSTON WORKSHOP 

I 

I 

! • QUALITY AND POLICY RELEVANCE OF FEDERAL LEVEL STATISTICS 

■ VARY WIDELY 

f 

i 

• FAILURE TO ANTICIPATE DATA NEEDS HAS LED TO POLITICAL 
PRESSURE ON STATISTICAL AGENCIES 

o PRIVACY PROTECTION AND BURDEN MINIMIZATION ARE 
PROBLEMS OF INCREASING SIGNIFICANCE TODAY 

e OFFICE OF FEDERAL STATISTICAL STANDARDS AND POLICY 

(OFFSS&P) CRITERIA ARE APPROPRIATE TO NATURAL RESOURCES 
STATISTICAL SYSTEMS, BUT NEITHER CONCEPTS NOR 
INSTITUTIONS ARE SUFFICIENTLY DEVELOPED TO PERMIT TRADEOFF 
AND COST BENEFIT ANALYSIS COMPARABLE TO THOSE UNDERTAKEN 
FOR ECONOMIC AND OTHER FEDERAL STATISTICS 

• REMOTE SENSING MAY BE A USEFUL DATA SOURCE, BUT ANY 
CHANGE IN EXISTING DATA DATA/INFORMATION GENERATION 
SYSTEMS WILL ONLY OCCUR IF THE SOURCES AND MAGNITUDES 
OF THE ERRORS ASSOCIATED WITH THE NEW PROCEDURES ARE 
FULLY CALIBRATED 

• REMOTE SENSING CAN SERVE IN SOME CASES TO MAKE LOCAL 
STATISTICS MORE ACCURATE 

• FOR MANY RENEWABLE RESOURCES CLASSICAL STATISTICAL 
TECHNIQUES ARE 'NEARLY APPLICABLE'. 'NEARLY APPLICABLE', AS 
MOST ASSUMPSIONS UPON WHICH CLASSICAL TECHNIQUES ARE 
BASED. ARE VIOLATED IN APPLICATION 


-F38- 


PRELIMINARY SUMMARY AND CONCLUSIONS FROM 
HOUSTON WORKSHOP (con't) 


T 

• WE MUST CONTINUE WORK ON DEVELOPMENT OF ROBUST METHODS^ 
FOR STATISTICAL ASSESSMENT OF RENEWABLE RESOURCES 
APPLICATIONS 

• CASE STUDIES EMPLOYING THOROUGHLY VERIFIED TEST DATA SETS 
WOULD BE OF GREAT VALUE IN NEW SAMPLING PROCEDURE 
DEVELOPMENT 

t- WE MUST UNDERSTAND ACENCYS' OBJECTIVES OR THERE CAN BE 
NO BONA FIDE ANALYSIS 

o IN MANY CASES IT MAY BE PREMATURE TO ASSESS BENEFITS AS 
USERS ARE SCIENTISTS ASSOCIATED WITH GOVERNMENTS 
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* PRELIMINARY RESEARCH AREAS FROM HOUSTON WORKSHOP 

I 

I 

I 

© DEVELOP CONCEPTS TO FACILITATE TRADEOFF AND COST BENEFIT 
I analyses of economic and OTHER FEDERAL STATISTICAL 

SYSTEMS. THESE STUDIES SHOULD ADDRESS QUESTIONS OF: 

i 

- Privacy 

- Accuracy Verification 

- Costs 

o ANALYSIS ORIENTED TOWARDS MAKING SATELLITE SENSORS 
'MATURE MENSURATION SYSTEMS' (AFTER ROSENFELD) 

• USE OF REMOTE SENSING TO IMPROVE THE ACCURACY OF LOCAL 
STATISTICS 

• NEW APPROACHES TO SAMPLING MORE RESPONSIVE TO VARIATIONS 
FROM NORMAL CONDITIONS 

• NEED FOR BASIC UNDERSTANDING OF RESOLUTIONS NECESSARY 
FOR THE PRODUCTION OF MINIMALLY SUFFICIENT STATISTICS 
FOR GIVEN APPLICATIONS 
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PRELIMINARY RESEARCH AREAS FROM HOUSTON WORKSHOP 
Ccon't) 


• NEEL FOR CASE STUDIES TO IMPROVE OUR UNDERSTANDING OF 
TODAY'S STATE-OF-THE-ART-ESTIMATION SYSTEMS. THIS APPROACH 
MAY REVEAL CAPS IN OUR KNOWLEDGE 

• NEED FOR CONTINUED FUNDAMENTAL EXAMINATION OF SAMPLING 
THEORY AS APPLIED TO RENEWABLE RESOURCES TYPE 
INFORMATION PROBLEMS 


RESEARCH AREAS AS DEFINED BY DISCUSSIONS WITH 
OTHER INDIVIDUALS 
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• WORK ON THE FUNDAMENTAL ASPECTS OF ACCURACY 

• METHODS FOR MAKING SATELLITE SENSORS ‘MATURE MENSURATION 
SYSTEMS FOR GIVEN APPLICATIONS AREAS 

• DEVELOPMENT OF LINKS BETWEEN RESEARCH, ARTIFICIAL 
INTELLIGENCE, DATA BASE, AND IMAGE ANALYSIS LOGIC AND 
THEIR IMPACTS ON 'EXPEPT SYSTEMS' 
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